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It is the purpose of this paper to attempt, an evaluation 
of the forces operating on the public schools which might in- 
fluence their disposition toward the utilization of technology. 



TECHNOLOGY IN THE PUBLIC SCHOOLS? 



INTRODUCTION 

Public education is in trouble. In the face of the ever-rising 
costs of operating our schools and demands of teachers' unions for 
high.^r salaries and increased benefits, school districts across the 
country are confronted with something of a taxpayers' revolt. The 
taxpayer himself daily contends with rising prices and increased taxes 
over which he seems to have no control. It is not surprising that 
public scrutiny should focus on the schools, for school bond issues 
and tax elections are among the few opportunities for the taxpayer 
to have some personal influence over who gets his tax dollar and how 
much. And it is not surj)rising that he says "Nol'^ when he takes a 
clojjer look at what he's getting for his money. 

Students are coming out of schools ill-equipped to join the labor 
force; businessmen complain that graduates are functionally illiterate. 
Inrer-city and rural kids fall further and further behind on achievement 
tests and/or drop out; suburban kids cry for relevancy and are increas- 
in'jly dropping out. 

It has been suggested that in the wide-ranging technology and know- 
how generated by our multi-billion dollar investment in research and 
development in such government agencies as the National Aeronautics and 
S)ace Administration, the Department of Defense, and the Atomic Energy 
Commission lies a fund of knowledge that can be adapted and applied to 
the solution of such public problems as those found in education. It is 
•proposed that large-scale utilization of available technology and skills 
would produce better results more efficiently. 
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It is the thesis of this paper that the answers to such questions as 
whethe:-, when and how technology is utilized in a society are contingent 
upon t ie social and political conditions prevalent in that society. It is 
furthe r suggested that three major developments in our society have great 
bearin'j on how these questions will be answered with respect to education: 
1) the very fact of the development of impressive technologies in the DoD, 
NASA, ind AEC, and their interest in finding and encouraging civilian 
application of their discoveries; 2) the current predicament of local school 
distri:ts, pressured by calls from the public and from state and federal 
authorities for pedagogical and fiscal ^'account ability" — encouraging the 
receptivity of educators to the idea of utilizing available technology and 
skills to provide better education more efficiently; 3) a behaviorist 
learning theory which supports, even demands, the systematic use of 
techncilogy to improve education. The convergence of these- .thx^.e developments 
in our society today constitutes a powerful force conducive /:i;p£ the large- 
scale utilization of technology in education. v-.'^w.- 

Chapters I, II, and III are devoted to delineating the manner and 
extent to which these three factors are manifested and interrelated, and 
the potential significance of their combined impact, that potential being 
illustrated and elaborated upon in Chapter IV via an ideal type case in 
poir.t. 

Chapter I notes a growing public awareness of and appreciation for 
the development of impressive technologies in the Dob, NASA, and AEC 
thi'ougli the use of systems analysis, complemented by a concomitant and/or 
consequent interest on the part of officials within these agencies and 
potential beneficiaries (including educators) to adapt and apply these 
t-3chnologies for civilian usage, e.g. Air Force developed programmed 
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instruction and simulation techniques, NASA interest in potential educational 
uses of computvsr technology and communications satellites. 

Special emphasis is placed on the -'technology'' of systems analysis 
itself, a highly sophisticated problem solving technique used as an aid 
to decisionmakers by quantifying the consequences of alternative courses 
of action. For we find that systems analysis is. being put forth as a 
model for the solution of social problems (e.g. education), particularly 
in the form adopted by the DoD. the Planning-Programming-Budgeting System 
(PPBS) . Systems analysis in this form involves defining program objectives 
and suo-objectives in operational terms and grouping resource requirements 
and activities by objective. Such a technique facilitates the analysis 
cf alternative strategies in terms of their relative effectiveness in 
meeting specified objectives and in terms of their costs, both current and 
projected into the future. We observe that the Rand Corporation, once 
prima?'ily concerned with doing systems analysis for the Air Force, is 
now e^igaged in applying forms of systems analysis, including PPBS, to the 
solution of civilian problems, including those in public education. More- 
over, we are cognizant of a suggestion to use monetary incentives to 
encourage the use of systems analysis. 

In view of the original success of the quantitative methodology of 
systsms analysis in improving man-machine systems, I suggest that educators 
influenced by the model of systems analysis, will be disposed to consider, 
the use of developing technologies and to think of the education enterprise 
in terms of man-machine systems. 

The survey of the accountability movement in Chapter 11 reveals that 
the bulk of this rather broadly based movement involves attempts to improve 
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the fimctioning of school bureaucrac: js* and that it tends to dovetail 
with l.he influence of systems analysis and PPBS--insof ar as virtually 
every aspect of the accountability movement addresses the issues of 
costs and/or effectiveness. Implicit, and often explicit, in the call 
for accountability is an appeal to ^'systems" thinking. Discountented 
tax-pf-yers, parents, students, legislators, and other critics and 
educators are demanding that material and human resources (input) be 
related to and determined by desired results (output). A corollary 
to this notion is the demand that every contributor to the school exper- 
ience be held accountable for the effectiveness of his contribution. 
And inplicit in this notion is the use of incentives. Effective contri- 
butioT'S merit reward. Others do not. 

While the accountability movement is still in an embryonic stage, 
the influence of systems analysis and PPBS is apparent in the efforts of 
educators to be responsive to criticism and demands Beyond the (usual) 
initi.il decision to individualize instruction for increased effectiveness, 
progrims vary in their emphasis on one or more aspects of the process, 
rangiag from rudimentary stnacturing of resources and activities around 
operationalized objectives to sophisticated techniques of cost-effectiveness 
analysis and assessment of accountability, stressing pedagogical and/or 
fisc£^.l accountability. We do note that the validity of. many of the appli- 
catic»ns of quantitative methodologies to education is under contention, 
both in the measurement of student performance (output) and in the 
rela^'-ive contribution of resources (input). 

* 

Tnree peripheral "movements" are discussed in Appendix A. 

Some of the issues involved in the question of valid assessment are dis- 
cussed in Appendix B. 



Nonetheless, effox'ts proceed to improve measuring instruments and 
methodologies; and while they do, major school districts are taking 
steps to implement PPBS and many state legislatures are either mandating 
or considering its use in the public schools. 

In our sampling of programs of individualized instruction, we do 
find educators utilizing educational technologies, conventional media 
and computer applications. The latter is used in computer-assisted 
instruction (CAI), primarily for the presentation of programmed in- 
struction, and computer-managed instruction (CMI) which facilitates 
record keeping and program structuring but can also bring to bear a 
wider array of data available on students than would normally be utilized 
systeir.atically. Ultimately, the contribution of educational technologies 
will te subjected to cost-effectiveness analyses which may well determine 
the extent of their future application. 

Chapter III points out that the influence of systems analysis and 
the uiilization of incentives and technology in the accountability move- 
ment are buttressed by a behaviori5.t learning theory, itself a form of 
systems analysis and the foundation of programmed instruction. Emphasized 
is t^.e fact that B, F. Skinner, the recognized "father" of programmed in- 
struction, believes that anything can be taught, if we define the terminal 
behavior (desired results) in operational i zed terms and base the structure 
of contingencies of reinforcement (positive incentives) on careful analysis 
of *.:hat behavior, but that the efficient arrangement of such contingencies 
is a sufficiently complex and demanding task to virtually require the 
utilization of technology. We perceive that the notion of programming 
instruction by breaking down a task into small units and rewarding 
appropriate behavior is a powerful and growing force in education, 
particularly among those utilizing educational technology. 

ERLC 
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Chapter IV examines an ideal type case in point, OEO's one-year 
"test'' of performance contracting, illustrating the convergence of 
those forces operating on trhe public schools (discussed in the pre- 
ceding chapters) that might be conducive to their use of technology, 
and offering a glimpse of the potential significance of such a con- 
vergence . 

OEO's "test" was designed to evaluate the relative cost-effectiveness 
of existing techniques of teaching offered by educational technology firms; 
firms ^vere held accountable for meeting established objectives; incentives 
were u:ilized at every level; all firms utilized programmed instruction. 

Analyses of effectiveness* discern no significant difference between 
experimental and control groups, with some few exceptions. Unfortunately, 
there Is no indication of what makes those exceptional cases significantly 
differ3nt from their control groups, i.e. the relationship between resources 
expended (input) and student performance (output). The more sophisticated 
techniques of economic analysis reveal that some experimental programs 
cost less than their conventional counterparts and that some of these 
were ]ess time-consuming — of no small import, especially to low-achieving, 
slow -earning students, the prime target of accountability, in general, 
and performance contracting, in particular. Of primary importance to this 
study is the fact that those programs which were most economical proved 
to be those which spent less on certified teachers and more on parapro- 
fessionals, materials, and equipment. 

V/e call attention to the results of Rand Corporation researchers in 
other areas which "imply that the development of techniques for instructional 
system design can radically improve the effectiveness of instruction 




k(;eping in mind the difficulties of measurement in education and its 
ejabryonic stage of development. 
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in almost any context" (emphasis added) and find that 

Rand is developing a computer program to examine the use-^^of 
communic^^tions media systems in instruction. . . .This program 
w: 11 develop ways of integrating media systems with other 
ei'.ements of instruction based on the criteria of cost, 
practicability, and conformity with instructional strategy. 

(Pincus, 1971:11-12) 

And we suggest that future cost-effectiveness analyses of educational 

techno] ogies may be highly attractive to school district personnel , 

faced v'ith demands for fiscal and pedagogical accountability, and 

r 

instituting PPBS or in states that are doing so. 

Chapter V deals with the expected opposition of unionized certified 
teachers to the implications of cost-effectiveness measures and incentives, 
and particularly to the use of technologies which may ultimately cost them 
their jobs. /It considers their current public support, based on distorted 
facts, and concludes that, given the public preference for better manage- 
ment and cost-cutting, the dissemination of more accurate infonnation may 
undercut that support. Such cost-efficiency and cost-effectiveness informa- 
tion are expected from the newly established National Institute of Education, 
whose Tiission includes research and development designed to increase our 
ability to use technology and media effectiveness in education. Knowledge 
generated by the NIE (or elsewhere, e.g. Rand) could feed into PFB systems 
being established in school districts and states--and gather support from 
the fs.ct that local districts are dependent upon states for a sizable 
portion of their income, states (perhaps, supported by federal agencies) 
which may offer incentives for the utilization of the results of their 
research. 

We do recognize the distinction between conventional audio-visual 
materials ;ind large-scale systems of instructional television or computer- 
assisted instruction, focusing "on the dearth of software available for the 
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latter. We conclude with the possibility that--based on the future 
foreca.sting of PPBS analyses--given the economic pinch school districts 
are in, they may. well engage in some form of joint or centralized decision- 
making to create a sizeable enough market to encourage the production of 
softwa-.:e. And we may well see such large-scale systems. 

Ch.'ipter VI is an attempt to explore the social and political 
implications of the large-scale introduction of technology into education 
under ':he conditions delineated in preceding chapters. For we noted in 
Chapter I the dependence of systems analysis on quantitative methods and 
its relatively greater success in improving man-machine interactions. 
And we suggested then that those applying systems analysis to education 
might well be influenced to conceive of the education enterprise in terms 
of man -machine systems. We must here take heed that we do not become 
so fascinated with the machine that we lose sight of the nature of man. 

Emphasis is placed on the particular importance of this point, given 
the behaviorist learning theory (itself a form of systems analysis) 
undergirding so much of the movement toward the utilization of technology 
in education and the foundation of the programmed instruction for which 
that technology will be used. For however much advocates of progranmied 
instritction assert that their students are actively involved in the learning 
process, that "activity" is always in response to stimuli carefully struc- 
tured to shape their behavior. They are engaged in a thoroughly passive 
activity, because behaviorist learning theory views man as a thoroughly 
passive creature totally shaped by his environment. He is completely 
malleable — putty, if you will, in the hands of his controlling environment 
and he who shapes those controls. 

Other theories see an inherently active, creative side to man--a vital 
need for freedom, bound to resist all forms of control--recognizing repression 
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in conr.rol. We are, of course, talking about values --but not merely 
values. For if Mead and Freud and Marx represent a more valid notion 
of the nature of man, we are talking about repressing a vital need. 

Tlie possibility of repressing a human need for freedom through the 
large--; cale use of a more effective educational technology demands that 
we ask questions about the nature of the decisionmaking about what will 
be tauj'ht and who will be teaching it (who will be writing programs toward 
what er.d ). For contrary to the assertions of people like Skinner and the 
creatois of a surveillance and detection system of CMI, we cannot make 
decisions in the interest of our culture. Our culture is made up of people 
who haA'e different interests. 

Tl.e dangers of overlooking the diversity of interest in our society 
is exemplified by the quantitative methods which are the essence of systems 
analyses and PPBS — which define a system operating in an environment, the 
limits of which are beyond control--and responsibility. When systems 
analyses attach numbers to sociological factors and the computer spews out 
the alternatives, we cannot forget the political implications of the power 
of those analysts over the fate of real people. We cannot be blinded by 
the mystique of the machine, be it CAI, CMI, or large-scale social analysis. 

People make decisions and we must assure that access to decisionmaking 
remain wide open--and that no large-scale system be mandated. 
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I. HAVE TECHNOLOGY- -WILL APPLY 



Can the ^'systems approach,'* which has helped build America's 
m:.ssile and space power, be put to work effectively in the rejuven- 
a-.:ion of American public education, so sorely in need of curriculum 
rt^form, new and imaginative plants and equipment, and daring redefi- 
Hction of its purposes in an era of satellites and civic strife, 
Computerization of the workaday world, and the well publicized revo- 
lution of rising expectations in the poverty-stricken **other America"? 

The answer to this overwhelmingly important question is not yet 
a'^ailable because the question has only recently begun to be asked, 
Aiaong the most persistent questioners is the Aerospace Education 
Foundation, an affiliate organization of the Air Force Association. 
T'\ 1966, the Foundation adopted as a priority goal the exploration 
of the potential of the new educational technology--much of it de- 
veloped by the Air Force through the use of the systems approach — 
for the enhancement of America's civilian schools- 

Toward this end, the Foundation, on a year-round basis, has 
been working with the U, S. Office of Education and with local and 
state school officials around the country on .a number of projects 
designed to extract ideas and adaptations for civilian use from the 
Pit Force's store of new and proven space-age instructional techniques 
end systems. These range from learner -paced programmed teaching to 
use of multi-media courses that use simulation techniques, which can 
subject the student to real-life environments in which the consequences 
of the student's decisions and the depth of his understanding of the 
subject matter are immediately clear to the learner, (Leavitt, 1968; ix) 

William Leavitt ' s expression of support and enthusiasm for adapting 
tecnnology developed by the Air Force for use in civilian schools were made 
in the Preface to Technology and Innovation in Education, a volume prepared 
by the Aerospace Education Foundation, based on an annual seminar held in 
cooperation with U, S. Office of Education. Tlie seminar was devoted to 
"the call to action in the job of putting already available educational 
technology to work in America's schools." It was characterized by 
Dr. R, Louis Bright, then U. S. Associate Commissioner of Education for 
Research, as having "more intellectual power than has ever before been 
mobilized for an exercise of this kind." (Leavitt, 1968:x) 

Such statements and the very existence of this volume, and others 
like it, are indicative of a growing commitment to channel space/military/ 
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nucleir technology into the civilian sectors of our society v/here it can 
be usod to solve the problems .of private industry and applied to the public 
probl '^ms of government agencies, thereby broadening the benefits of an 
otherwise restricted research and development effort. 

Rr.chard Lesher's study of effective means for channeling these new 
technologies in promising directions emphasizes that *'a wide range of kinds 
of technology will be transferred, including inventions, discoveries, develop- 
ments modifications, systems and techniques'* (Lesher, 1966:66). In his 
discursion of how technologies with potential civilian application might 
be idrmtified, Lesher (1966:155) notes that NASA provides a model in the 
NASA t)ffice of Technology Utilization. 

The Space Act of 1958 charged NASA with the obligation to 
♦'provide for the widest practicable and appropriate dissemination 
of information concerning its activities and the results thereof." 

In response, NASA has evolved a program.; under an Assistant 
Administrator for Technology Utilization, to identify new technology 
resulting from the agency's broad rangirr^ R§D programs, to report it 
(where practical J in industrial terminology, and to communicate it to 
organizations in the civilian economy through several mechanisms, 
including regional dissemination centers. (Lesher, 1966:117) 

Of :our5e, much of this technology will be of little use to local public 

school districts, but even the most obvious --the .use of satellites to 

de]iver education--implies the possibility of large-scale use of technology. 

Th'3 feasibility of utilizing satellites for this purpose is at least 

partially dependent upon wide-spread use of such technologies as instructional 

television and computer-assisted Instruction. 

S ystems Analysis 

Beyond application of any particular type of technology, Lesher 
notes Sumner Myers' attention to a more fundamental kind of transfer. 

ER?C 
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He sees such activities as the space program setting new standards 
of achievement for the entire technical community. He asserts that 
"the space program may be stimulating the process of technological 
innovation by changing professional norms and general attitudes." 
He suggests that "the very existence of the space program as a model 
of technological achievement may prove more important to the economy 
than either the multiplier effect of its investment or the spillover 
of its technology." 

Noting that "the chief factor making for innovation in a community 
is prior innovation," Myers contends: "Perhaps the most pervasive 
contribution of the civilian space program may turn out to be the 
strength it has given, at the firm level, to those who push for 
innovation. The people who are for innovation now have more signifi- 
cance and have stronger arguments than those who oppose innovation. 
This is not only true at the firm level but is also encouraging 
people to push for bolder social undertakings. IVhether they approve 
of the particular goal of the space effort or not, they use it as 
a model of how things might be done — from curing cancer to rebuilding 
cities. (Lesher, 1966:67) 

The use of the space effort "as a model of how things might be done" 

involves the utilization of the problem solving capability developed in 

th(i military/space sphere, methods for which Lesher finds insufficient 

encouragement in the civilian sectors .of our society. 

But, David Allison and others have suggested: "The most important 
derivation of this (the military/space) R§D effort is likely to be 
a new ability to solve problems. Not strictly technical problems, 
but those involving a mix of components: Technical, managerial, 
psychological, social, political." (Lesher, 1966:69) 

This problem solving capability is a highly sophisticated systems 

analysis approach which, though used earlier, came into its own during 

Wcrld War II when scientists became involved in developing methods for 

plotting weapons (e.g. radar) systems into operation. L. Eugene Root 

di.scusses a definition of operations analysis that emerged from the 

military experience: 

"Operations analysis provides quantitative bases for management 
decision." Four things should be noticed about this definition. 
First the analyst, by implication, is not the manager and does not 
hii.iself make the decision, at least in his role as analyst. Second, 
the information supplied to the manager is quantitative. It attempts 
to indicate in some numerical form the consequences of various possible 
decisions and thus to clarify for the manager the consequences of his 
action . . . 
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In the third place, there is the old dileiiuna that 
quantitative recommendations are no better than the input data 
or. which they are based. 

The fourth point relating to our definition postulates 
the existence of someone who can take all the relevant inputs 
ar,d make a decision. (Root, 1970:4-5) 

Root rotes that systems analysis 

ir the ver>' nature of its dependence on quantitative methods has 
been most successful in fields which have a large technical con- 
tent and a rather smaller emotional or psychological content. 
Operations analysis in wartime dealt with improving man -machine 
interactions — finding out the best way to use new and complex 
devices. (Root, 1970:6) 

Nonetheless, he urges us to attempt the application of systems analysis 
in the quest for solutions to social problems. I would submit that 
schoolmen, pressed for solutions to their own growing problems and 
tumirg to systems analysis for assistance, may well be influenced to 
consic.er among their alternatives the use of developing educational 
technologies and to conceive of the education enterprise as something 
of a man -machine system, 

.boot's comments introduce The Challenge to Systems Analysis: 
Public Policy and Social Change, a collection , of papers suggesting the 
application of the systems approach to such problems as urban development, 
commiinications, population control, the world food problem--and computer- 
assisted instruction. The paper by J. A. Stockfisch on the introduction 
of systems analysis into the Department of Defense is suggested as "a 'case 
in point' that has general relevemce wherever man seeks to use instruments 
of che state to achieve objectives" (Stockfisch, 1970:8). It is proposed 
as a model to help executives manage bureaucracies efficiently,, and 
Stockfisch specifically mentions local school boards among the possible 
beneficiaries of these methods. As schoolmen are, in fact, turning to 
systems analysis for aid in coping with their problems and since proposals 
for action are increasingly made in terms of the systems approach, we 
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we would do well to examine the model rather carefully. 

Stockfisch begins his discussion with a review of the problems 
involved in managing a bureaucracy, emphasizing the distinction between 
the role of the "executive*' and that of the ^'operator. The executive 
is the policy maker, the specialist in ends; the operators are the doers, 
the specialists in means who make up the line agency, the bureaucracy. 
The desire of the line agency to maintain its autonomy leads it to engage 
in a viriety of activities to elude the direction of the executive, not 
the least of which are its efforts to keep him in ignorance of its 
activi*.:ies. Any effort on the part of the executive to interfere with 
the ac;ivities of the line agency, to *'rock the boat," to change policy, 
is met with resistance cOid hostility. 
Stockfisch summarizes: 

1. Thiere is a lack of useful operating knowledge at the executive 
level. . . . 

2. As a results of condition 1, the executive office is unable to 
determine whether in fact its policy goals are being implemented. 

3. Because it cannot clearly determine i^hether its goals are being 
implemented, the executive has no objective check of whether his 
stated goals may be mutually consistent or whether they are even 
feasible. Without knowledge of whether they are feasible, it is 
hard to know what their cost of attainment will be. Thus not 
only may the executive office be in the dark, but also the 
administration as a whole may wind up kidding itself .... 
Finally, there is no measure or way of knowing, even in the 
case when a policy goal is being achieved, that the goal is 
achieved in the most efficient or least costly manner. 

(Stockfisch, 1970:16-17) 

In order to cope with the problem of gaining knowledge about the 
operations of the Department of Defense upon which to formulate consistent, 
feasible policy goals and to determine whether they were achieved effi- 
cien^Jy, Secretary of Defense McNamara and his staff instituted two 
inno/ations. The first has come to be called the planning- programming- 
budgeting system (PPBS) . The second created a systems analysis staff. 
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Defense policy goals involve systems and units which cut across 
service line agencies--**£or example, combinations of strategic bombers 
and Polaris submarines or of Army divisions, tactical Air Force 
squadrons, and Naval carrier strike forces. Organizations and units 
are drav^m from all three services and j)laced under a single and unified 
comDand.** (Stockfisch, 1970:17) The traditional practice of each 
sery ice submitting separate budgets itemizing its separate needs for 
research and development, personnel, procurement, operation and main- 
tenance, and installations was incompatible with the decision makers' 
neeo. to know the cost of a given system or program. By devising a 
budgeting system that begins with identifying policy objectives in terms 
of ihe programs or "mission responsibilities" cutting across service lines 
and then determining costs on the basis of input, from whatever source, 
the policy maker is in a position to know the cost of any given program 
(and its elements) designed to perform a specified function. Moreover, 
by using this budgeting system, it is possible to project the costs of 
a program and its elements over a period of years, thus making It possible 
to see the implications of current decisions for the future. 

Stockfisch emphasizes the advantages of being able to identify 
systems designed to perform specified functions. For given the defined 
ends, or policy objectives, a systems analysis staff is employed to 
utilize operations research to analyze alternative mixes of elements or 
irputs of the system and to utilize economics to specify and quantify 
e'.'fectiveness criteria. Thus, means can be related to ends in such a 
wiy as to aid policy makers to make rational decisions in terms of 
effectiveness and efficiency. Cost-effectiveness studies have become 
Ihe basis upon which new programs or program changes are justified. 
CStockfisch, 1970:17-19) 
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Systems analysis was the primary concern of the Rand Corporation 
in ii s work for the Air Force. Rand analysts are now applying their 
methc'ds to aid in the solution of civilian problems, e.g. law enforcement, 
public health--and education. At the 26th Joint Study Group on Military 
Research Allocation Methodology, David Levine presented a paper on 
Structuring Program Analysis for Education Research (Levine, 1970) and 
Polly Carpenter presented A New Kit of Tools for Designing Instructional 
Systems (Carpenter, 1970). In fact, as we shall see. Rand analysts have 
done a considerable amount of research and evaluation of the problems of 
education. Many of their analyses and proposals for solutions are 
summarized in Policy Studies at Rand: Education and Human Resource s 
(Pincus, 1971). 

Lesher has suggested that the most pervasive "technology" to come 
out cf the military/space sphere is systems analysis, should application 
of s\*ch an approach to managing our school bureaucracies demonstrate 
the tiffectiveness and/or efficiency of educational technologies, relative 
to alternatives, its influence could indeed be pervasive among educators 
pressed to find solutions to their problems. 

Incentives 

Pursuant to the notion of systems analysis is a suggestion proposed 

at is\ Engineering Foundation Research Conference on "Technology and its 

Social Consequences." 

The suggestion involves local competitions for government 
grants to design systems solutions to urban problems. Patterned 
in part after the Atomic Energy Commission's requests for pro- 
posals on the location of its proposed new linear accelerator, 
the suggestion would be for the Federal Government to offer a 
sizable grant~or matching funds ---to the winner or winners of a 
competition for the design of systems for mass transportation, 
waste disposal, and other urban problems. (Lesher, 1966:143) 

Q As we shall see, pressed by demands for more effectiveness and efficiency 

ERIC 
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--ac':ountability, educators are receptive to the use of systems analysis, 
competition for funds (the use of incentives), and technology as possible 
solutions to their problems. 
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II. ACCOUNTABILITY— DO IT! 

There is a rather pervasive ''accountability movement" in education 
toda/ whose aims are strikingly similar to those Mr. McN^mara had in mind 
when he instituted systems analysis and PPBS in the Department of Defense. 
The novement is permeated with the notion that schoolmen are "accountable** 
for ictuaily doing what they are supposed to be doings-educating our 
youtn-~and doing it effectively and efficiently. And so, people are 
engaged;, in a variety of ways, in attempts to evaluate what the schools 
are doing and to formulate alternative approaches to operating the schools 
to achieve policy objectives. 

What Is It? 

The notion of accountability in education is not a new one. It has 
dou1)tless been around as long as teachers. In ancient Greece, except for 
the Sophists who taught for a fee, most education in the polls took place 
informally in public discourse — at the marketplace, in the gymnasium, in 
public assembly. When a man like Socrates gained a reputation for his 
wisdom, he attracted a circle of followers; and thus, in part, were the 
youth instructed. However informal the discourse, such a man was accountable 
to the polls. And, indeed, when Socrates' unconventional teachings and 
criticism of Athenian democracy were adjudged to be sacrilegious and 
ard corrupting the young, he was sentenced to death. Since then, of course, 
our concept of education has become increasingly formalized and our notion 
of accountability increasingly precise and specific (though, to be sure, 
the piuiishment less severe). Roger Lennon reminds us: 

ERIC 
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At the University of Bologna in the 15th century, student-enacted 
i-tatutes required that the "professor start his lectures at the 
beginning of the book, cover each section sequentially, and com- 
jilete the book by the end of the term"; should any professor fail 
lo achieve the schedule, he forfeited part of funds that he him- 
i elf had had to deposit at the beginning of the term. 

And then: 

The concern of governmental bodies that they were getting their 
educational dollar's worth is hardly new. In 1911 the Board of 
f'stimate of the City of New York, critical of the demands made 
ty the Board of Education on the city's treasury, launched a 
survey of the city's schools, one aspect of which was an analysis 
of the tested arithmetic achievement of its pupils. (Lennon, 1971:4) 

Callahan (1962) has studied the "cult of efficiency" "^ich grew out 
of Frederick W. Taylor's time and motion studies in the nation's steel 
mills. During the 1910's and 1920' s, education "engineers" and "experts" 
appeared on the scene urging the adoption of Taylor's "scientific management" 
and attempting to apply his methods and use of development capital, outside 
audits, analysis of process, establishment of standards, incentive pay — 
all designed to. relate factory input to output most efficiently. Unfor- 
tunately, a good deal got lost in translation. They failed. IVe are left 
with a legacy of red tape, mountains of record keeping, rigid standards, 
and burely a trace of relating input to output. Callahan suggests that 
they vere more interested in economy than efficiency and that they had 
neither the research skills nor the money necessary to be successful at such 
an undertaking. 

Today's accountability movement, partly inspired by the success of 
systems analysis and the use of incentives in our military and space efforts, 
echoes many of Taylor's concepts. We may well have the necessary research 
skills; we certainly have considerably more research money than available 
fifty years ago. 

But accountability today means many things to many people encompassing 
a broad range of interests and issues. It is generally a striving for ways 
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aiid means of affixing, assuming and meeting- responsibility for the education 
of 0U3- youth. And *it is rooted in a pervasive and often intense discontent 
with the functioning, or malfunctioning, of our schools today. 

F rim^ari i >v ^ f rom our suburban students, we hear cries for "relevance'^ in 
protest against rigid standards reflected in sterile, inflexible curricula and 
the monotony of the trivia directed at them daily. They complain that they 
can find no relationship between what goes on in their classrooms and what 
they see and hear and know about the real world around them." Tiiey are demandinj 
a clesr relationship between what they spend so many hours of the day on, so 
much cf their lives, and what is truly important to them. 

A bit of the ^^relevancy'* theme can be found in ,the inner city, where 
minority groups are demanding that greater recognition be given to their 
uniqu'3 characteristics and their contributions to our society. But ''relevance^' 
can be icing on a cake they do not have. Their problems are grave. Burt and 
Lessihger (1970) quote former U.S. Commissioner of Education, James E. Allen: 

One of every four students has a, significant reading deficiency. 

In large city school systems up to half the students read 
below expectation. 

Half the jobless youth, in the 16-21 year old age bracket, 
is functionally illiterate- -this illiteracy represents a barrier 
to success and produces lives marked by poverty, unemployment, 
alienation, and in many cases., crime. (Burt and Lessinger, 1970:144) 

It is here that the accountability movement picks up steam. The fact 

tha^. so many of our youth are not receiving even the most basic education 

pro/okes first dismay and then indignation, not only among the minority poor, 

but in society at large. And it is here that demands for educational account- 

abrlity become linked to demands for economic . accountability, that links are 

made between costs and effectiveness. State legislators, pressed by the 

increasing costs of education and requests for funds, and congressmen used to 

tliinking in terms of "more bang for the buck'* are asking questions that 

schoolmen are having difficulty answering: ■ IVhere is all this money 



goin^? For what? They are demanding not just lists of materials and 
services purchased but an accounting that relates resources expended to 
program objectives — output. Likewise taxpayers are regularly refusing 
the requests of school boards for the passage of bond issues and 
increased taxes. 

The sticky question then becomes:* Who is accountable to whom and 
for v/hat? Alkin (1972) has suggested that three basic types of accoun- 
tability can be discerned: 1) goal accountability--in which the school 
board is accountable to the public for goal and objective selection; 2) 
program accountability--in which school district management is accountable 
to the school board for the development and/or selection of instructional 
prog'iTams appropriate for stated objectives; 3) outcome accountability-- 
in which the instructional manager (i.e. the teacher) is accountable 
to tlie school district management for producing program outcomes con- 
sist 3nt with pre-selected objectives at a performance standard appropriate 
for the instructional program. 

This may well be a good place to start, but it is insufficient to 
truly take cognizance of all the parties involved in the education enter- 
prise or of the complexities of authority, responsibility, and power in our 
schools. Hough (1971) points out that state statutes hold school boards 
alore responsible for the education of our youth but that teachers' unions 
have demanded and in some states received legal sanctions, and school 
boards find themselves legally charged to negotiate with teachers over 
a variety of issues concerning salary increases, fringe benefits^ job 
secirity, and policy decisions. School boards, solely responsible for 
the operation of our schools, are in a spot. Conversely, teachers object 
to being held accountable for the outcome of programs without a sizable 
voice in policy decisions. 
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Moreover, state departments of education have ruled, notably with 
respt:ct to contracts let to private performance contractors (to be discussed 
belovO> that school districts may not delegate authority to make policy or 
respc»nsibility for management. These same departments of education enforce 
states regulations concerning school policy. 

Students would like more influence in the formulation of school policy, 
and it has been suggested (Campbell, 1971) that the students themselves be 
held accountable for their own performance — with the acknowledgement that 
such a proposal would entail changes in compulsory attendance laws. 

Lennon (1971) and Deterline (1968) have discussed the notion of holding 
deve!»opers and publishers of instructional materials responsible for the 
performance of students using their products and programs, Lennon points 
out *.he difficulties involved in guaranteeing to produce specific amounts 
of learning when producers have so little control over the use to which 
their materials will be put and so much depends on teacher competency and 
the structure of the total instructional system, Deterline acknowledges 
that guarantees of effectiveness are an impossible requirement to make 
of any instruction material to be used as one component in an instructional 
setting dependent on a teacher pulling together materials, media, and 
methods--attempting to construct a program based on materials without 
spe::ified objectives or procedures for their use. But he asserts that 
guarantees are '^exactly what the entire educational materials design and 
publishing field has always needed,'^ He maintains that programmed 
instruction can provide such warranties and that all well designed 
materials should systematically include validated specifications of 
objectives and quality control procedures. 

Accountability, then, is the object of a rather broadly based move- 
mc^nt in public education which tends to dovetail with the influence of 
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systoms analysis and PPBS insofar as virtually every aspect of it speaks 
to tie issues of costs and/or effectiveness. Implicit, and often explicit, 
in t?ie call for accountability is an appeal to "systems" thinking. It 
involves an attempt to include all participants in the educational process 
in an endeavor to relate output (student performance) to input (financial, 
mate:rial, and human resources). The relationship should specify the con- 
tribution of each resource to meeting specified objectives. Moreover, 
implicit (and often explicit) in the notion of accountability is the uti- 
lization of incentives. Effective contributions merit reward. Others do 
not. If the role of teachers and administrators can be clearly defined 
and their performance in these roles assessed, they can and, indeed, 
should be rewarded on the basis of performance — in terms of salary increase, 
promotion, prestige. Conversely, failure to perform entails "punishments." 
Likewise, incentives are evident when people like Deterline suggest that the 
day is coming when schools need not purchase, instructional materials without 
gua:rantees. If, in fact, chey did not and guarantees were available, 
developers and publishers of materials would be forced to compete for 
regards (purchases) or suffer punishment (non-purchase). 

I suggested above that educators, turning to systems analysis for 
as:;istance, might well begin to conceive of the education enterprise as 
a nan-machine system and to favor alternatives involving the use of 
educational technologies. As the accountability movement is concerned 
with much the same issues as systems analysis, one would expect it to 
fivor educational technologies — insofar as they can be demonstrated to 
contribute to the effectiveness and/or the effic^ ^ncy of the operation 
of schools. Likewise, insofar as educators feel threatened by the 
accountability movement, one would expect them to oppose the use of such 
technologies. 
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There are three '^movements" on the fringe of the accountability 
movenent which are less specifically concerned with systematically relating 
inpu'i: to output. Those people interested in promoting educational vouchers, 
free schools, and community control find the source of educational ills 
in the bureaucratic structure of our schools per se--within which they 
feel the needs of our youth cannot be met. These people propose alter- 
native solutions to what they consider to be the problems of centralized 
administration of large city school systems and/or the uniformity of 
prescribed curriculum and instruction. From their respective points of 
view, these people go beyond holding schoolmen responsible for doing a job 
well to doubting it can be done--within the confines of the current structure 
of school bureaucracy. For further discussion of these "movements," see 
Appendix A. 

Individualized Instruction 
The bulk of the accountability movement accepts the overall structure 
of t/ie schools (being perhaps less skeptical or viewing themselves as more 
pragTiatic than the fringe "movements") and attempts to make that bureaucracy 
work. Efforts to be more responsive to criticism and demands have led to 
greater concern for individualizing instruction — to meet the different 
neecs of individual students. There are increasing numbers of "alternative" 
schools being opened — the public schools' answer to free schools. These 
schools are intended to serve the needs of students inhibited' by the tra- 
ditlonal classroom. But the core of the accountability movement focuses 
on 'Manipulating and assessing what goes on in that class room- -and affixing 
responsibility for output. 

The literature is replete with proposals for and reports on 
efl'orts to systematically relate input to output and individualize instruction 
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withxn the regular classroom as a means of doing so. A look at a repx'e- 
sentative sample can give us a better idea of what they are about, 
Struc:turing Programs Around Specified Objectives 

A cooperative Service Agency in Wisconsin is operating a Title III 
project, entitled ''Individualizing Instruction Through Media--Rural, '* 
encompassing thirteen rural districts with 92 teachers, 31 paraprofessionals, 
26 clerical assistants, 2760 students. Pupils work on contracts which 
specify objectives and materials suited to their individual needs, pre- 
pareo by teachers trained to prepare instructional objectives in behavioral 
terms, design instructional sequences to individualize learning, and use 
medio. A wide array of media is utilized, particularly in the areas of 
language arts, science, and math, including the fo].lowing hardware (with 
appropriate software): 16mm film projector; filmstrip projector; individual 
filmstrip viewers; slide projector; overhead projector; tape players, with 
headsets and jackboxes; cassette tape recorder, with headsets and jackboxes; 
language master. Much of the software is made available through the 
Insti'uctional Materials Center at La Crosse via a truck route serving each 
school twice a week. There are also facilities available at each school 
for ?.ocal materials production. Each student is instructed in the operation 
of all equipment which is then made available to him for lise on his par- 
ticular contracts. Pre-test performance determines assignment of contracts, 
whica in turn specify the criterion performance of required proficiency 
necessary on post-tests before proceeding to the next skill or concept 
(next contract). The project, enthusiastically received by students, 
parents, teachers, and supervisors, has had a significant impact on learning, 
as evaluated by the Wisconsin Department of Public Instruction after the 
firsc year of operation. (Solberg, 1970) 
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The Learning Research and Development Center of the University of 
Pitti^burgh, funded by the U. S. Office of Education, has developed 
Individually Prescribed Instruction (IPX) which, with a few variations, 
look*', much like the Wisconsin rural project. It is designed to provide 
for i.ndividual differences in children through the use of a variety of 
instructional settings, programmed materials, self -instruction, and multi- 
media modes of instruction. Like the Wisconsin project 

The developmental model for IPI considered the following aspects 
of instruction as they related to the individual: (1) detailed 
specification of educational objectives; (2) organization of 
methods and materials to attain these objectives, including a 
variety of paths for mastery of any given objective; (3) a pro- 
cedure for the diagnosis of student achievement in terms of the 
educational objectives; (4) individual daily evaluation and 
guidance of each pupil, including a system for individually 
prescribing the learning task that the student is ready to 
undertake; (5) provision for frequent monitoring of student 
performance in order to inform both the pupil and the teacher 
of progress toward an objective; and (6) continual evaluation 
and strengthening of curricular and instructional procedures. 

(Scanlon and Brown, 1971:95) 

Like the Wisconsin project, IPI makes use of teacher aides to help 
with grading and paperwork. The main differences seem to be that IPI does 
not use contracts with students (which tend to run from three to five 
days) but rather evaluates pupil progress daily; that IPI relies less 
heavily on media; and that although the IPI teacher daily uses data 
to prescribe individual learning tasks, self-instructional materials and 
teaching sequences were apparently organized by the Learning Research 
and Development Center. 

With the cooperation of Research for Better Schools, Inc., IPI 
has been field tested in five demonstration schools, serving different 
student populations, e.g. disadvantaged, rural, special education, 
Indians, Mexican-Americans. Scanlon and Brown (1971:104) report that 
students and teachers have positive attitudes toward the system and 




th:it "IPI students achieve as well or better than non-IPI students on 
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stanuard tests These demonstration schools have hosted thousands of 
visitors interested in individualized instruction. To help meet the 
needs of those interested in establishing IPX in their schools. Research 
for Better Schools, Inc. and the Learning Research and Development Center 
have devised a strategy for the training of administrators and para- 
prof c ssionals, the retraining of teachers, and the monitoring of these 
schoC'ls to assess the degree of implementation of IPX and the progress 
of students. 

Scanlon and Brora report that the costs of IPX have been decreasing, 

citing a 200% reduction in mathematics materials (without, unfortunately, 

specifying what those materials are or comparing the cost to non-IPX 

schoc Is) . But they note that 

Mass adoption of the IPX system will necessarily involve 
publishers and other commercial sources. State laws in many 
cases will have to be modified to accommodate the IPX system. 
State approved textbooks and purchasing procedures will also 
need modification. The purchase of consumable items, the 
payment of teacher aides, and the provision for retraining 
funds are some of the other obstacles that must be overcome 
if Individually Prescribed Instruction is to be available 
on a national scale. (Scanlon and Brown, 1971; 104-S) 

Computer-Managed Instruction 

Progranis of individualized instruction, such as the Wisconsin 

"Individualized Instruction Through Media — Rural" and IPI, involve 

assessing the needs of individual students, specifying behavior 

objectives appropriate to those needs, prescribing materials and 

procedures designed to meet those objectives, monitoring student 

pro'^ress toward objectives--all of which requires the systematic collection 

analysis;^ and display of data on student performance and appropriate 

materials for prescription. Such a task can consume large amounts of time 

on the part of teachers and/or paraprofessionals. 

ERLC 
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Some programs are making use of computers for data-processing, including at 
least one IPI school. Valuable staff time is then freed to attend to the 
needs of the students, basing instructional decisions on the diagnostic and 
prescriptive information provided by the computer. Moreover^ the use of 
computers facilitates the consideration of a wider array of data on student 
charcXteristics, beyond current academic performance. (Silberman, 1969) 
John C. Flanagan (1968), Chairman of the Board, American Institutes 
for Research, and Verne S. Atwater (1972), President, Westinghouse 
Learning Cor^Doration, have reported on the cooperative effort of their 
resptJCtive Institutions in the development and distribution ^ computer- 
managed instruction (CMI) system called the Program for Learning in 
Accordance with Needs (PLAN). Flanagan reports that 

Project PLAN represents a systems approach to educational problems. 
By "systems approach'^ it is intended to indicate that all the rele- 
vant factors in the situation are considered and not just selected 
aspects. In other words, the whole problem rather than some portion 
of it will be treated. (Flanagan, 1968:113) 

Project PLAN is designed to utilize resources currently available to 

schools serving all types of students in a program of individualized 

instruction based on guidance and individual planning to aid the student 

''to develop plans which will enable him to prepare himself for those roles 

thr.t he wishes to play in society'^ (Flanagan, 1968:115). It encompasses 

thi teaching of mathematics, language arts, social studies, and science 

for grades one through twelve.. There are five major components of 

Project PLAN; 

1) The computer processes data to be made available to teachers and 
administrative personnel to aid in decisionmaking. 

2) Behavioral objectives specify changes in behavior to be produced 
(terminal behavior), usually grouped in two-week modules, indi- 
cating to student and teacher what the student is to learn during 
that period of time in any given subject matter. 



-29- 



l) Performance standards (criterion tests) correspond to the 

behavioral objectives. 
•I) A guide, the teaching-learning unit, is given to the student and 
teacher, indicating the objectives, materials to be used and how 
to use them, and how to check for the achievement of desired be- 
havioral changes. The guide is intended to provide suggestions, 
but the student is not obliged to follow them. He must merely 
demonstrate learning by meeting performance standards. 
13) Guidance and individual planning units and tests determine the 
student's abilities, his learning style, his plans and potentials 
for future activities and roles. Those, combined with occupational 
information integrated into the instructional program, help the 
student to understand himself and learn to make realistic choices. 
In twelve years of schooling, a student will have chosen about 1200 
of 5000 available modules. Of course, this decisionmaking is 
informed by the data-processing function of the computer, which 
(in addition to scoring, record-keeping, scheduling) compares 
student and materials and recommends short- and long-range steps, 
providing alternative teaching-learning units suited to the student' 
learning style. (See Flanagan, 1968) 
Atwater (1972) reports that, af^er three years of development and 
tef/ting in fourteen school districts with lOyOOO students. Project PLAN 
now involves some 20,000 students throughout the country. He notes that 
student interest increases with responsibility, teachers have a new sense 
of professionalism (training and consulting services being available), and 
acfminis trators "can account to parents and taxpayers for both the costs 
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education and the progress of individual students." Space and materials 
are more effectively used, and funds for coionseling and remedial work 
are freed for other purposes, as these activities are incorporated into 
the regular classroom. Moreover, the data-processing function of the com- 
puter facilitates the evaluation of the relative merits of specific 
teacher- learning units for given types of students, the value of computers, 
television arid other hardware (and software) available for instructional 
purposes. (Atwater, 1972; and Flanagan, 1968) 

John F. Cogswell (1966), of the System Development Corporation, has 
reported on an instructional management information system being designed 
in conjunction with their analysis of the Continuous Progress Plan (CPP) 
developed at Brigham Young University Laboratory School (grades seven 
through twelve). Their study of CPP was a part of research into the im- 
plementation of instructional media through systems analysis and computer 
simulation. 

CPP is based on the use of **Study Guides" which specify all required 
work for a given course. The student works individually on texts or 
progr?4iunod instructional materials obtained from the Instructional 
Mafarial Center. Teacher aides monitor study areas, providing routine 
assistance. Students needing further help file requests by describing 
their problems on forms which are analyzed daily to form homogeneous 
group5> for group-help sessions. Over time, the size of the groups 
decreases as the variation among students increases. Students determine 
when they are ready to be tested on a unit of study, file requests, and 
are assigned to the continuously operating Test Center. Those who pass 
progress to the next unit; these who do not do further work before re- 
testing. The latter happens (ideally) infrequently, a*? rJtudents set rate 
anl achievement "expectancies" (based on their past records) with the aid 
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o£ counselors. 

To keep CPP working smoothly, the instructional management information 
system designed by System Development Corpora»:ion keeps a daily record 
of all scheduled activities, ensuring that appointments are kept and 
thai equipment is ready, used, and rbtumed. But central to the data- 
processing function of the computer is the "surveillance and detection 
system.*' Teachers, aides, students, counselors, and administrators use 
teletype stations to insert the following information into the computer's 
student-information data base: 

1) Biographical data. 

2) Students' past performance; achievement test scores on course 
work and the dates of testing. 

3) Reports filed by teachers and aides on the student's interests; 
learning, emotional, and social problems. 

4) Requests for materials, classes, and extracurricular activities 
filed by the student. 

5) Student's schedules and '^expectancies" for course work. 

6) Records of counseling interviews. 

7) A long-term schedule of "major events" in the student's 

high school career, e.g. vocational-planning interviews, college 
placement discussions, etc. 

8) A record of the degree to which the student's current interests 
and activities are consistent with original goals set earlier. 

The information logged in the computer at any given time forms pre- 
dictive criteria against which student progress can be compared. Periodi- 
cally the computer scans the data, making judgments about the degree to which 
the student is meeting "expectancies." Discrepancies result in '*red flags" 
ini.erted in the student's file, signaling the need for attention, whereupon 
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the computer produces a display of specified problems which it routes to 

apprc'priate personnel (and possibly to the student, ^^containing carefully 

selected data")* Problems might include failure to take a scheduled test 

or to pass it, failure to appear at group sessions, or a discrepancy 

between current interests and activities and long-range goals set earlier. 

Aftei- the appropriate staff member meets with the student, he enters at a 

teletype station a record of the interview and its results, including any 

adjustments in schedules or expectancies. Should the computer find the 

student to be meeting schedules and expectancies, it merely records this 

information in his information file. However, it is felt that no student 

should go too long "without some fairly structured and regularized contact" 

with a staff member. If the computer finds no such recorded meeting, it 

sets a red flag on his file to alert the staff to the need for appropriate 

action. (Cogswell, 1966a and 1966b) 

Cogswell sees great potential for a fully developed surveillance and 

detection system: 

Such a system would be capable of special sensitivity to students 
whose difficulties were extreme or nearly continuous; in the files 
of these students, a single red flag would route a direct warning 
message to the appropriate counselor. . .The system would provide, 
and encourage the use of, easy ways for the students to express 
their own needs, reactions, and interests. In short> such a system 
would approximate the awareness and interest of a dedicate.d teaching 
staff whose whole "student body" consisted of a single pupil. 

(Cogswell, 1966a:101) 

Computer-Assisted Instruction 

We have observed that the call; for accountability has led to greater 
concern for meeting the different needs of individual students. Attempts 
have been made to set behavioral objectives in terms of the interests and/or 
neec-.s and abilities of the individual child. Some are utilizing the data- 
processing capability of computers for record-keeping, scheduling, and 



instrructional prescription in the management of individualized instruction 
--freeing staff members for individual attention to the counseling and 
tuto:."ial needs of students. Others are attempting to use the computer for 
the delivery of instruction itself. Gabriel D. Ofiesh, who played a major 
role in the development of the U. S. Air Force's programmed learning metho- 
dology, has defined computer-assisted instruction (CAI) thus: 

By "computer-assisted learning," we mean using a computer to 
present materials to human organisms by driving display devices, 
such as typewriters or projectors or tape recorders, etc., to 
help students understand concepts by performing problem-solving 
calculations and simulating real-world situations. 

It is assisting the student learning process and therefore is 
called "computer- assisted instruction" (CAI)...CAI is a man-machine 
relationship in which the man is a learner and the machine is a 
computer system with a purpose of inducing human learning and 
retention. 

Another point that should be made — it is fundamental and rather 
axiomatic- -is that computers can only process data that is put into 
them. Computers do not think, feel, or create new information. 

(Ofiesh, 1968:59) 

Hall (1971) has distinguished four uses of computers in education: 
1) Laboratory computing device: Students have direct access to the computer 
as a *cool to develop programs related to their course work, primarily in 
mathematics, physics, and chemistry. Hall estimates that there are some 
500 high schools in this country utilizing computers thus. 2) Record- 
keeper and retriever: The computer is used for administrative purposes and 
those we have delineated' for CMI. 3) Simulation: This use of computers 
has been centered in higher education in the field of medicine, where 
computers have been variously used to simulate "patients" for whom the 
student provides a diagnosis. 4) Tutor: St is this use to which the 
liteiature ms'ot commonly refers as CAT. The computer might be used for drill 
and practice exercise or it can become the primary source of instruction, 
providing sequential exposition of programmed materials. 

Hall notes the peculiar advantages of CAT for individualizing instruc- 
tion: A student working at a computer terminal is actively responding to the 
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material presented to him. It precludes the passive role of students 

exposed to traditional uses of texts (or oral or visual presentations)-- 

often cited as a particular problem of slow learners. While the student 

is actively responding to the material, a computer delivering a pre-stored 

program can evaluate and provide feedback to the student's response in* 

a matter of seconds: 

Results to date show that students receiving instruction from 
computers respond anywhere from once every four seconds to once 
cvery 30 seconds. This means that each student. . . is responding 
and receiving feedback from 40 to 600 times during a 40-minute 
session at a computer terminal. (Hall, 1971:628) 

The speed of the evaluative and feedback functions of the computer permits 

the iximediate alteration of a course of study in accordance with the 

immediate past history of a given student to achieve pre-determined 

criterion performances. (Hall, 1971:628-9) Moreover, Hall reports that 

>i consistent result in the use of computer-assisted instruction 
)ias been that the same amount of material has been learned in a 
CAI environment as in a conventional classroom, although with a 
considerable saving of time in favor of CAI. (Hall, 1971:630) 

Such a saving in time could be of considerable value, particularly to 

those* interested in improving the performance of today's low-achieving 

slow learner. 

With respect to costs. Hall maintains that the costs of CAI are 
comparable to those of conventional instruction and that rising personnel 
costs and decreasing technology costs make CAI an attractive .system to 
school administrators. He suggests that the possibility of utilizing a 
computer for daytime delivery of CAI to students, for in-service and adult 
education after school and in the evening, and for administrative purposes 
from midnight to 8:00 a.m. make the installation of such a system econo- 
mically feasible in many school districts and gives it a competitive ad- 
vantage over conventional instruction. (Hall, 1971:630-1) 
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Assessing Accountability 

Assuming that goals are defined, there are essentially two distinct 
aspects of assessing accountability. First, students must be tested to 
determine the extent to which their behaviors meet goals. Given appro- 
priate measures evaluating student performance, a determination must be 
made of the factors influencing that performance, to evaluate the extent 
of the influence of school programs and personnel. As yet, there are no 
well-dsfined agreed-upon procedures for performing either of these tasks-- 
to relite input to output. 

The primary issue in the testing of students concerns the nature of 
the te.5ting instrument, the implications of using one type of test as 
opposed to smother, and their relative validity under certain conditions 
for certain purposes. The determination of factors influencing performance 
on any given testing instrument is itself a complex task, partly because 
of issues relating to the testing and partly because. of the need to identify 
and quantify influences over which educators presumably have no control and 
for which they therefore have no Responsibility. These, ranging from the 
child's IQ and socio-economic status to community and school plant conditions 



would be held constant to isolate the influence 

i 



of school personnel and 



programs. 

Data generated from the use of such techniques should have valuable 
policy implications. Beyond the possibility of refining personnel selection, 
assignment and remuneration policies, such data should also lend itself to 
an evaluation of the relative cost-effectiveness of alternative programs of 
resource allocation--including the educational technologies. 

It is true that the entire area of testing and measurement--determining. 
what a student has learned and what factors account for that learning — is in 
sonewhat of an embryonic stage. But the controversy generated by the account- 
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ability movement has stimulated interest in it and will likely contribute 
to its development. For further discussion of the issues involved in testing 
and pioposals for determining accountability (some of which are being imple- 
mented) , see Appendix B. 

PPBS in Public Education 
Ke have noted that many of the issues involved in the accountability 
movement are not unlike those Mr. McNamara hoped to deal with when he intro- 
duced PPBS and systems analysis into the Department of Defense. Both reflect 
a concern with achieving specified objectives, determining the degree to 
which those objectives are being met and the cost of doing so. The goal 
is to assure that policies are being carried out effectively, and effi- 
ciently, and to have the knowledge at hand to make intelligent decisions 
on alternative courses of action. We might recall that Stockfisch commended 
the u'se of PPBS to executives charged with managing such bureaucracies as 
the. public schools. And, indeed. President Johnson was sufficiently im- 
presssd with the operation of PPBS in the DoD to issue a presidential direc- 
tive in 1965 mandating its use by all federal agencies — including the Office 
of Education. 

Since that time, there has been a spate of literature relating the 
advaitages of PPBS and its application to specific school problems. State 
legislatures, following the federal lead, have been most responsive--80% 
have either mandated or considered its use in the public schools. And 
some administrators have already taken steps to implement it in their local 
districts, including those in Los Angeles, Chicago, St. Louis, Philadelphia, 
Baltimore and Memphis. 

The Rand Corporation has been instrumental in developing plans to imple- 
ment PPBS in California (Farquhar, 1971). And Rand analysts have produced 
a series of papers on its pui-poses and methods and the exigencies of 
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applying it to public education. Haggart has delineated The Program 
Structuring Aspect of PPBS for Education (1971) and Carpenter has ex- 
plicated the Analysis of Educational Programs (1971) and Cost Effectiveness 
as an Aid to Making Decisions in Education (1970). The point is first to 
indicate the disadvantages of the traditional budgeting systems, which 
merely list ''instruction'* as a line item followed by a a lump sum--perhaps 
broken down into subjects taught --without supplying any information about 
what tiose figures mean in terms of what is happening in "instruction." 
By contrast, the advantages of PPBS are presented in terms of a program 
structure relating activities (and their resources) to measurable objectives 
these objectives broadly stating goals, are broken down into sub-objectives 
(in behavioral terms) of subprograms. The budget not only indicates the 
costs of what people are doing to achieve objectives, but projects those 
costs into the future. PPBS, then, is more than an accounting system. It 
provides decision makers with the information they need to evaluate and 
select from alternative programs on the basis of their cost-effectiveness. 

Putting PPBS into effect is. dependent upon the existence of a rather 
elaborate information system. And Rand analysts have produced a six volume 
study, designing such a system for the management of the Los Angeles school 
district. (Farquhar, et al ., 1971) 

One would expect that schoolmen putting PPBS into effect will make judg 
mentj. about educational technologies in terms of their relative cost-effect- 
iveness. 
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III. BEHAVIORIST LEARNING THEORY— CAN DO 



V.'e noted in Chapter I the existence of technology and the use of 
systens analysis and incentives in the military /space sphere. We. have 
seen the latter echoing throughout the accountability movement and the 
utilization of technology (conventional A-V equipment, CAI, CMI) to 
individualize instruction as a means of achieving accountability. All 
three of these influences on education are buttressed by a behaviorist theory 
of learning. 

In that chapter on the influence of our military/ space efforts, we 

made irention of a volume prepared by the Aerospace Education Foundation, 

entitled Te chnology and Innovation in Educatiop . It is fitting that the 

forward to this book should have been written by B. F. Skinner, whose early 

work included training ^^missile-guiding pigeons" for the Air Force in 

World War II. Skinner's name has since become synonymous with behaviorist 

learning theory. He is the recognized "father" of programmed instruction. 

].t is significant that the Air Force and the Aerospace Education 
I'oundation are working closely with public education. The historian 
(;f the future, in writing about education today, will undoubtedly 
note, and will probably be puzzled by, the fact that technological 
advances in education have been picked up much more rapidly by 
;».ndustry and the services than by our schools and colleges. 
There are some obvious explanations. 



But the reason most often given is a supposed distinction be- 
tween training and teaching. 

It is often said that industry and the armed services can use 
programmed instruction because they ar^ interested in training and 
that programmed instruction is therefore appropriate. But what does 
traiining mean? 

Tlie first step in constructing a program is to decide what^ you 
want to do. You must define the terminal behavior. What is the 
student to do as a result of having been taught? Only when you 
know that can you arrange conditions under which he will acquire that 
behavior. 
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In the armed services and in industry we usually know what we 
want the student to do, and we know when he has learned to do it. 

In schools and college the situation is very different. No 
one has defined the terminal behavior. No one has specified pre- 
cisely what the student is to do as the result of being taught. 

The distinction between training and teaching comes down to 
this: If you know what you are doing, you are training, and if 
you don't know what you are doing, you are teaching. (Skinner, 1968a: 

Skinner's behaviorist learning theory is grounded in a firm belief 
that man is^ his behavior and that behavior is shaped solely by what happens 
afte:* it takes place, by how it is reinforced. Behavior followed by nega- 
tive reinforcement will result in aversive future behavior. Behavior 
followed by positive reinforcement will tend to be repeated in the future. 
His work with animals, young children, and mental retardates has led 
Skimier to believe that it is possible to teach any behavior if we can 
define it, analyze it, and control the conditions under which it occurs. 
To critics who maintain that one cannot teach insight and creativity by 
programming. Skinner answers that if they would just define "creativity"-- 
how does a creative person behave?--the behavior could be analyzed and 
contingencies of reinforcement arranged to produce that behavior. 

The whole process of becoming competent in any field must be 
divided into a very small number of very small steps and rein- 
forcement must be contingent upon the accomplishment of each step. 

(Skinner, 1968b: 21) 

And then Skinner takes note of human limitations. 

These requirements are not excessive, but they are probably 
incompatible with the current realities of the classroom. 
In the experimental study of learning it has been found that 
the contingencies of reinforcement which are most efficient in 
controlling the organism cannot be arranged through the personal 
mediation of the experimenter. An organism is affected by subtle 
details of contingencies which are beyond the capacity of the 
human organism to arrange. Mechanical and electrical devices 
must be used. Mechanical help is also demanded by the sheer number 
of contingencies which may be used efficiently in a single experi- 
mental session. (Skinner, 1968b;21) 
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While there are those within the behaviorist camp who differ 
witn Skinner over the forms of programming — linear or branched — and 
those who do not limit themselves to the kind of teaching machine he 
specified, the notion of programming instruction by breaking down the 
task and rewarding appropriate behavior is a powerful and growing force 
in education. To those who would attempt to discredit programmed in- 
struction on the grounds that it has proven no better than conventional 
instruction. Skinner would answer that those programs were based on poor 
analyses of the behavior to be shaped and that better analyses would lead 
to better programs and schedules of contingencies of reinforcement--and 
better shaping of behavior. And, indeed, for many educators, this 
remains the ideal — particularly among those in educational technology. 
We have seen its influence on computer-assisted instruction. And the 
literature on instructional television indicates a similar influence. 
(Kaplan, 1972) 

The behavioral technology exemplified by Skinner's work promises to 
fill the knowledge gap made so obvious by the efforts of accountability 
advocates to design educational systems made up of programs relating output 
to input. For invariably we return to the same question: What should we 
be doing to improve student performance? 

Skinner reminds us that Sidney Pressey designed a machine that could 
teach in the 1920's, but without a concomitant behaviorist theory, the 
invention was essentially ignored. We might add that neither was there 
a vast technological and systems analysis capability developed by a 
military/ space effort for which civilian applications were sought. Nor 
\as there a climate of accountability, with forces at work in education 
•striving to relate inputs to outputs. Today there is. 
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IV. PERFORMANCE CONTRACTING- -TELLS ALL 

Much of the literature on accountability is concerned with 
better msinagement; specifying behavioral objectives, the performance 
of which can be demonstrated; developing measures of output (student 
behavior change) and measures of the degree to which school personnel 
meet their responsibilities. Less attention is given to determining 
what personnel should be doing in terms of process specifications.* 
Part of this is due to the early stage in which we find the accountability 
movement, and is the result of a conscious desire on the part of accounta- 
bility advocates to shift from a traditional emphasis on methodology to 
a new emphasis on output--not how the teacher teaches but what the student 
lec.ms. Unfortunately, it is not that simple. 

For instance, we can employ a technique comparable to Barro's suggestion 
of a multiple regression analysis in a given school district.** We might 
determine in a hypothetical situation that output measures (of student 
performance) are less than specified by objectives, but that the failure 
cannot be attributed to teacher performance. Rather the fault lies with 
the program of instruction and the supendsory personnel charged with its 
development. Such a discovery is certainly a valuable clue to administra- 
tors that they need to develop better programs. Still, one cannot help 
but step back and ask, **So, what else is new?" What have we learned that 
we did not already know? For was it not our failure to develop effective 



We might call this task operations research, as would be performed by 
a systems analysis staff evaluating alternative approaches to achieving 
given behavioral objectives; or behavioral analysis, as would be per- 
formed by a Skinnerian to construct appropriate programs of instruction 
and contingencies of reinforcement to achieve those objectives. 



r-n^r>' ** See Appendix B. 
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instructional programs (at least for low-income, low- achieving students) 
thai;, in part, generated this new accountability movement? 

In an attempt to cut through many of the problems attendant upon 
efforts to make accountability work and, indeed, to make Every Kid a 
Winiter , Leon Lessinger (1970) has become the leading advocate of 
per^rormance contracting. Acknowledging the difficulty of stating many 
educational goals in terms of behavioral objectives which are measurable, 
Lessinger stresses that we can do so for such basic skills as reading and 
arithmetic — which are, after all, the crucial areas in which low-income 
and minority students perform poorly. He advocates local educational 
authorities drawing upon outside talent and resources to engage in a kind 
of "educational engineering." His approach is designed to encourage low- 
risk experimentation in order to find that mix of instructional technology- 
hardware, software, incentives, methods — that produces specified results. 
While school districts across the country would benefit from the testing 
of a variety of programs, performance contractors would be paid on the 
basis of guaranteed results. 

But what exactly does performance contracting involve? In 
order to use it effectively, a local education authority forms 
temporary alliances with several outside agencies. First, it 
needs a source of funds earmarked for educational development. 
School districts that are both prosperous and farsighted can 
raise some of this money from their own local budgets, but most 
districts will depend heavily on grants from state and federal 
agencies. 

Second, performance contracting requires a management support , 
group that can help local officials specify exactly what educational 
results are sought; in writing a request for bids from firms willing 
to do the job; in negotiating with and evaluating these firms; in 
drawing up the contract; and in dealing with the chosen firm. 

Third, the local authority needs the services of an independent 
educational auditor who will assess the children before and after 
the program and make a public report. This report will determine 
whether the contractor has met all of the requirements and, if so, 
whether he is entitled to the incentive payments for exceeding the 
minimum standards. 
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Last, but hardl/ least, the plan requires that various firms 
or other groups bid vigorously for the contract, tailoring their 
resources to the stated needs of the local school. 
(Lessinger, 1970: 18-19) 

A number of school districts have elected to utilize performance contracting 
to tjxperiment with new programs in an effort to find that combination of 
resources which will be successful with students who have traditional]/ 
bee)i failed by our schools. After the much publicized first contract 
was let in Texarkana, the Office of Economic Opportunity implementeci a 
one -year ''test*' of performance contracting in the schools, "the largest 
field experiment conducted in a single year in the history of public 
education in the United States." (Blaschke, 1971b: 1) 

I have attempted to delineate in this paper certain forces operating 
on the public schools which might be conducive to their use of technology: 
the availability of that technology, systems analysis (including cost- 
effectiveness), the use of incentives, the accountability movement, 
behaviorist learning theory. It is possible to Mew OEO's test of perfor- 
mar.ce contracting, perhaps the most extreme response to calls for fiscal 
anc* pedagogical accountability, as the embodiment of these influences. 
The experiment itself was a form of systems analysis, designed to evaluate 
the relative cost-effectiveness of existing techniques of teaching offered 
by educational technology firms. Technology firms were to be evaluated on 
the basis of their ability to provide the alleged advantages of performance 
contracting, including 

Improving the reading and math skills of poor, under-achieving 

children through the use of incentive-based contracts. 

Reducing the costs of increasing a child's achievement by 

certain grade levels. 

Effecting institutional change by introducing new techniques 

and instructional devices into the cla ssroom, and by developing 
an awareness among school officials of the need to establish 
educational objectives and determine whether those objectives 
are being met. (OEO Pamphlet 3400-5, 1970:6) 
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Whi le performance contracting is, by definition, incentive-based, OEO 

utilized incentives from the inception of the experiment, insofar as 

participating school districts and educational technology firms competed 

for inclusion in the experiment. While OEO was influenced by the need 

to assure representation of the major types of low-income, low-achieving 

pop'ilations in the country, to be rewarded with selection 

the school districts had to meet the following criteria: 

--Designate elementary and junior high schools for the experiment that 

met the criteria for assistance under Title I of the Elementary and 

Secondary Education Act. 
— Have at least 200 children each in grades, 1, 2, 3, 7, 8 and 9 

(100 for the experimental group and 100 for the control group) 
— Be able to provide data on student achievement and to provide space 

and personnel for the experiment. 
--Indicate that it anticipated no legal or political obstacles to 

mounting the experiment. 



This criterion was reH,/Uced to 75 students in three cases, 
to ajjow small, rural districts to participate in the 
experiment. (OEO Pamphlet 3400-5, 1970:7) 



Eij;hteen school districts were chosen: four large urban systems, nine 
middle-sized urban systems, and five smaller rural systems. They repre- 
sented poor Whites, Blacks, Chicanos, Puerto Ricans, Eskimos, and American 
Indians, 

Of the 31 technology firms responding to the OEO's request 
for proposals, six were selected on the basis of their corporated 
experience and interest in performance contracting, the types of 
achievement they thought they could guarantee, the qualifications 
of their staff, and the variety they represented in terms of their 
instructional approach (i.e., emphasis on hardware, incentives, or 
curricular software and teacher training methods). 

(OEQ Pamphlet 3400-5, 1970:8) 

Fach company was assigned to work in three demographically varied districts 
among the eighteen. Payment was based on a guarantee of a minimum level 
of improvement in each subject in each grade for each individual child, 
not class or site averages. Additional payments were to be made for im- 
provements beyond the minimum. Some contractors passed on the incentives 
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to teachers using some form of merit pay.* All but one contractor ex- 
tended the use of incentives to their students, whose contingency manage- 
ment included reinforcement in the form of tangible gifts, money, free 
time, games, movies, trips for free hamburgers and milkshakes. While a 
variety of resource mixes were demonstrated, all contractors utilized 
pro.^rammed instruction; and all employed paraprofessionals to a greater 
or lesser degree. (Blaschke, 1971b) 

OEO let separate contracts to the Battelle Memorial Institute to 
provide the educational achievement evaluation and to Education Turnkey, 
Inc. for management support. 

Two sets of tests were used in the experiment, one for determining 
the private firms' pay and one for OEO's evaluation purposes. Three 
different, nationally normed standardized tests, one of which was 
selected on a random basis for each class, were used for determining 
about 75 percent of the firms' pay, with the remainder of the pay 
determined by students' performance on criterion, or curriculum, 
referenced tests. A fourth standardized test was used only for 
evaluation purposes. (OEO Pamphlet 3400-5:11-14) 

Stringent measures were taken to preclude the possibility of "teaching 
to the test," safeguarding results obtained from standardized tests. Recog 
nizing the difficulties involved in measuring educational growth, ** Battelle 
evaluated differences between experimental and control groups on pre- aJid 
post-tests in reading and arithmetic, fitting data for each grade/site 
combination to a regression model based on raw scores (not the grade- 
equivalent scores upon which payment to contractors was determined). 
Tlieir summary of results for each grade/subject/site combination indicates 
t'lat there were 31 significant positive impacts, 54 significant negative 
impacts, and in 127 cases, there was no significant difference between 
experimental and control groups. 



The contractor operating in Texarkana, not included in this experiment, 
had offered stock in the company among its teacher incentives. 
** See Appendix B. 
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It is unfortunate that, given the intent of the experiment to test 
a variety of combinations of educational resources, no attempt was made 
to relate the various programs to the six performance contractors and 
their specific combinations of elements to the measured output. Why 
did Seattle have no cases of positive impact, 9 of negative impact, and 
3 o: no significant difference; whereas ^Anchorage had two cases of positive 
impact, none negative, and ten of no significant difference; and Dallas 
and Jacksonville both had six, zero, and six, respectively? What is it 
about the programs in third grade arithmetic that produced more negative 
impacts than any other grade/subject combination? What is it about the 
proj^rams in seventh and eighth grade arithmetic that can explain the fact 
thac they succeeded nowhere in achieving a positive significant difference, 
but that five such differences are found in third grade arithmetic? 
(See Battelle, 1972:85) It would be most valuable to have an analysis 
relating specific combinations of inputs to the measured output, especially 
in "chose cases where significant positive impact occurred. Unfortunately 
all we can say is that, in most cases, there was no significant difference 
in 3ducational output between experimental and control groups. 

More useful are the results of Education Turnkey Systems' economic 
analysis, indicating that fourteen of the grade/subject/site experimental 
progrcims had lower costs per student-year than their corresponding control 
programs. Of these fourteen, only three actually had lower costs at rates 
per hour of instruction than corresponding control programs, the remaining 
eleven having consumed less instructional time. (Blaschke, 1971b: 181) 
This raises rather interesting questions about cost-effectiveness: If 
the.' eleven programs achieved no significant educational difference in less 
tine, would they have done so in the same time? Are they more effective 
foe having achieved no significant difference in less time? Time is no 



-47- 



minor factor for students in compensatory education programs because 
they are slow lieamers. Such data on costs and effectiveness are of 
particular interest in light of Education Turnkey System's final 
comnents on. their economic analysis: 



In general, the experimental programs were able to demonstrate 
the following; 

lower classroom costs through better scheduling 

lower staff costs through the use of paraprofessionals* 
and more intensive scheduling 

reliance on individualized instructional systems (with 
a concomitant increase in audio-visual hardware and 
software cost). 

All of these characteristics have two things in common: 
the qualities of management control and flexibility. These 
qualities allowed each subcontractor to more effectively use 
the educational resources that went into each program. This 
improved management of scarce resources is possibly the most 
important prerequisite for realization. of any benefit from 
the advainces in educational technology. And these advances 
include, not only sophisticated hardware systems, but also 
the behavioral science advances as reflected in this experiment. 

(Blaschke, 1971b: 182) 

The Rand Corporation has done a separate evaluation of performance 
contracting for the Department of Health, Education and Welfare in a 
series of eight case studies (including only one site from the OEO 
experiment). Their findings, based on preliminary reports of disputed 
gain scores indicate respectable but not dramatic results. And, corro- 
borating Education Turnkey System's economic analysis, they find that 



The report later notes that there were some "states which requi:red 
that the classroom be under the supervision of a regular licensed 
teacher** but the contract program provided by the Office of Economic 
Opporunity did not specify their use. In such cases, the subcontractors 
were obliged to hire such staff although they were not needed, thus 
increasing their costs and distorting somewhat the purpose of the 
experiment. . (Blaschke, 197lb:201-2) 

See Appendix B. 
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(alchough performance contracting costs more than conventional instruction) 
when measured against comparable compensatory educational programs, perfor- 
mance contracting costs were the same or less. Moreover, they report that 
tho;>e performance contractors who spent less on certified teachers and 
more on paraprofessionals , materials, and equipment were .the . cheapest , 
(Carpenter and Hall, 1971) 

The data on . performance contracting is not inconsistent with ^ 
Donaldson^ s (1971) review of the literature indicating no significant 
difference between conventional instruction and the use of instructional 
tel'jvision, teaching machines, and programmed instruction. And Pincus^ 
review of Rand studies in educational technology includes research done for 
the Air Force in which 

the Rand team is developing tools for designing instructional 
systems and analyzing the impact of varying designs on the 
resources required for, and the costs of, instruction. Early 
results have shown that substantial savings in the costs of 
instruction are possible with no decrease in training effect- 
iveness. (Pincus, 1971:11; see also, Bretz, 1971) 



There is the notable exception of the mandated instructional 
television system in Hagerstown, Washington County, Maryland. 
The effect of ITV there has been remarkable. During the first 
year of its use, grade 5 students gained an average of 1.9 years 
in knowledge of arithmetical concepts. "In junior high school 
general mathematics, the average level of urban pupils ... 
rose in four years . . . from the 31st percentile«.X!P, .'^standardized 
test of concepts to the 84th percentile . . . and on a standardized 
test of problem-solving from the 33rd to the 68th percentile.'* 
Of particular interest are results of an experiment directly 
comparing the ITV system with classroom-only instruction, indi- 
cating that children of less than 90 IQ (average 83 IQ) gained 
only 6 months in achievement growth in a year with classroom-only 
instruction but 13 months with ITV, (Schramm, Coombs, Kahnert 
and Lyle, 1967:68-69) Of course, the results here are strikingly 
different from the bulk of the literature reporting no significant 
different between itV and conventional instruction (teacher --led, 
classroom-only). The task remains to determine what makes 
Hagerstown different and/or develop other instructional programs 
equally effective. 
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Morsover, Pincus continues: 

These results imply that the development of techniques for 
instructional system design can radically improve the effec- 
tiveness of instruction in almost any context , . , .Unfortunately, 
it has. .become clear that those planning and designing in- 
struction have only crude means at hand. 

To help remedy this situation. Rand is developing a computer 
program to examine the use of communication media systems in 
instruction designed for any type of subject matter or student. 
This program will develop ways of integrating media systems with 
other elements of instruction based on criteria of cost, prac- 
ticability, and conformity with instructional strategy. 

(Pincus, 1971:11-12, emphasis added) 

Reduced costs, even provided no significant difference in output, 

is an attractive feature of educational technology to school districts 

hardpressed for funds. Moreover, education delivered through such 

media as instructional television and computer-assisted instruction 

can reach considerably more students than any given **good teacher. '* 

Should instructional systems using communications media be developed 

which "radically improve the effectiveness of instruction,** the use 

of such technology will be especially attractive. And, the more 

widely utilized the technology, the cheaper it becomes. It would seem 

that school districts, faced with demands for fiscal and pedagogical 

accountability, perhaps instituting PPBS or located in states that are 

doing so, would be strongly influenced to embrace educational technolo] 
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V. COUNTERVAILING FORCES AND SOLUTIONS: 
POWER AND MONEY "WILL DO? 

The accountability movement has not been well -received by teachers. 

To 'Che contrary, a recent Teacher Opinion Poll reported by the NEA 

Research Division indicates that the closer to home accountability 

strikes, the greater the teacher opposition. Tne report reveals that 

48% of teachers polled opposed performance contracting, 71% opposed a 

voucher plan, and 88% opposed accountability payment, "whereby teachers 

would be paid on the basis of their pupils' achievement." TTie NEA 

researchers conclude that 

the results of this survey strongly suggest that public 

school teachers in general do not believe that the type 

of competition for money customary in the business world 

should be applied to education. (NEA Research Division, 1971:13) 

If we posit that responsibility is meant to accompany competition 
for funds in the accountability movement, and if we substitute the word 
responsibility for "competition for funds," we find that as responsibility 
moves from an outside agency to the school and then to the teacher, 
teachers' temperatures rise. 

But we should not be deceived by the 48% opposition to performance 
contracting, for contractors are employing practices, including the merit 
pa>' receiving 88% opposition, which pose a grave threat to teachers. 
The Grand Rapids Educational Association, although otherwise cooperative 
with the contractor in its district, strenuously objected to the use of 
merit or incentive pay — in this case, bonuses for some teachers. (Siimner, 
19 71) The Gary Teachers Union had a number of grievances, including the 
increase in pupil/teacher ratio, the use of paraprofessionals (who they 
maintained served as teachers), and differentiated staffing and salaiy 



rat.^s amounting *^to a hidden merit or incentive pay system." (Hall 
and Rapp, 1971) 

Indeed, Myron Lieberman, noting that 70% of the teachers in this 
country were covered by collective negotiations in 1971, suggested 
among developments likely to predominate in the collective negotiations 
of ':he 1970' s the following: 

* An intensive effort to organize paraprofessioftals in education. 

* Greater negotiating and legislative emphasis upon organizational 
security, especially agency shop clauses. 

* A growing concern with perforinance contracting, voucher systems, 
and other institutional changes that appear likely to undermine 
traditional employment relationships in education. (Lieberman, 
1971:215-216) 

In fact, the American Federation of Teachers has already flatly 
rejected performance contracting and the NEA has made acceptance of 
contracting and other forms of accountability contingent upon teacher 
sel f -governance. (Williams, 1971) 

It is not surprising that an educational technology that could 
reduce costs through greater reliance on equipment and paraprofessionals 
and less on expensive certified teachers should generate teacher oppo- 
sition, especially during a time of marked teacher surplus. Thus, 
William Graybeal, assistant director of the NEA Research Division, dis-* 
cussing the prospect of having three teachers on the labor market for 
every available job by the fall of 1&?2, suggests that we would have a 
teacher shortage, if we concentrated on such goals as raising the quality 
of education by decreasing the pupil/teacher ratio. (Graybeal, 1971) 
This despite the fact that there is no evidence to indicate that pupil/ 
neacher ratio makes any significant difference in student performance. 
It should be noted, however, that the cost-effectiveness measures demon- 
strated by performance contractors did not preclude individual attention 
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to pupils* needs; to the contrary, many contractors drastically reduced 
pupil/staff ratios by increased employment of paraprofessionals, sparing 
the cost of expensive certified teachers necessary to reduce pupil/ 
teacher ratios. 

Nor is it surprising that NEA's bid for self-governance should 

make the following point: "Board standards for employing qualified staff 

can be developed. These standards might reflect a 'vdgher level that the 

min'.mum certification requirements of that state" (Williams, 1971:60)^ 

Thii. despite the fact that Hanushek finds in his study of The Value of 

Teachers in Teachin g that 

the present set of hiring practices leads to^n inefficient allo- 
cation of resources. The analysis indicates that teaching experi- 
ence and graduate education do not contribute to gains in student 
achievement scores. Moreover, the characteristics that do matter 
are not highly correlated with these factors. Yet these attributes 
are being purchased by the school district. (Hanushek, 1970:25) 

Distorted information is one of the tools used by line agencies to 
elude the direction of executives and impede attempts to change policy in 
bureaucracies, discussed by Stockfisch (1970) in his paper on the intro- 
duction of PPBS into the Department of Defense. The educational system 
seems not to be immune to such practices. Stockfisch mentions another 
tactic, which has apparently proved successful for teachers--the development 
of a constituency independent of the executive. Thus, Charles Blaschke's* 
analysis of a recent Gallup survey reveals a not so curious array of senti- 
ments . 

The Gallup survey found that John Q. Citizen feels that the most 
important problem in public schools for 1971 is finances and cutting 
school costs. By a margin of 49% to 28%, he also favored performance 
contracting, because it introduces efficiency and accountability — 
"no results, no pay." By a margin of 54%, he also favored school boards 
hxring management "experts" to find areas where school costs can be reduced. 
(31aschke, 1971a:245) 



* of Education Turnkey System, management support group for OEO's experi- 
ment in performance contracting. 
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Yet the teachers have made their case with the average citizen, for he 
includes among rejected suggestions for cutting costs both the reduction 
of the number of teachers by increasing class size and the reduction of 
tea*:hers* salaries by a fixed percentage (which could be accomplished by 
hiring paraprofessionals or less experienced teachers)--neither of which 
would endanger pupil performance. (Blaschke, 1971a) 

But inasmuch as the teachers' constituency sets its priorities in 
favor of better management and cutting costs, teachers are ultimately in 
a vulnerable position. Insofar as it can be demonstrated that the use of 
instructional technology is more efficient than costly and unproven certi- 
fied personnel and this information can be made widely known to the public, 
Joh:i Q. Citizen is likely to strengthen his support for better management. 

The National Institute of Education could prove to be a source of such 
knowledge of cost-efficiency, as well as the more attractive cost-effective- 
ness foreseen by advocates of the technology. For among the suggested 
missions of the NIF; is research and development designed to increase our 
ability to use technology and media effectively in education, including: 
inr»tructional uses of the computer; cassette television; games and simu- 
lar.ions; and instructional environment. (Levien, 1971:89) 

Moreover, there are other conditions which might facilitate the use 
of technology. The President's Commission on School Finance (1972) not 
orly endorses the role of the NIE to generate and disseminate knowledge 
ox the potential value of educational technology, but further suggests 
t'lat **state governments establish state-wide evaluation systems to measure 
the effectiveness of educational programs'' and "that class size standards 
?.nd pupil/teachers ratios be used sparingly and selectively in the prepara- 
•:ion of school budgets and the allocation of staff until and unless further 
research indicates conclusively otherwise" (Pres, Comm. on School Fin., 
ERJC 1972:58;59). 
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The reconmendation that states assume the major burden of financing 
local school districts has recently been deemed (legally) unnecessary 
by the Supreme Court. But the fact remains that local districts are 
deprjndent upon state aid for a sizable portion of their income — which 
the/ would. not readily forfeit. We have already noted the interest of 
sta-ie legislatures in PPBS. Indeed, we have noted the same interest 
on zhe part of local districts. To the extent, then, that local dis- 
tricts remain dependent upon state agencies and grants from federal 
agencies, and these agencies, extend tendencies to require competitive 
bidding, perhaps specifying that proposals incorporate programs utili- 
zing the findings of research on cost-effectiveness (something local 
districts might be inclined to do anyway), the likelihood that we would 
ultimately see the large-scale use of technology in education would be 
considerably increased. 

# # # 

It should be noted that one does need to distinguish between readily 
available conventional audio-visual materials and large-scale systems of 
instructional television and computer-assisted instruction. The Human 
Resources Research Organization (HumRRO) is currently engaged ia a study 
of the factors which might account for the slow rate of diffusion of CAI. 
Un'oil the results of their study are available, we should note that, in 
order for systems of CAI (and ITV) to be economically feasible — and in- 
deed cheaper than the conventional system of education, which is dependent 
upon expensive credentialed teachers — they must be able to provide an 



The effect of the new program of federal revenue-sharing on this pro- 
position remains to be seen. 
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array of programs (software) to a large number of users. David Berkman 
(1972) presents the problem for CAI* essentially as follows: 

Few schools now have the hardware for CAI. It is very expensive to 
develop quality programs of instruction to deliver via computer* It 
would be extraordinarily expensive to develop curriculum-wide systems of 
CAI, What learning company is ready to invest such large sums of money 
''to develop curriculum-wide computer-based instructional systems, when 
no schools have the hardware through which CAI can be offered? What 
school is ready to commit the millions in capital equipment dollars 
required to purchase the computer hardware for which programming does 
not exist?" (Berkmn., 1972:458) Berkman suggests that if schools were 
in a tight enough vnou^tjy pinch, they would be forced to use the ultimately 
cost-effective CAI. 

A recent cost of education index indicates that despite evidence 
that schools expenditures have increased considerably since World War II 
(by over 250% in the last decade), increases for 1971-72 were, for the first 
tim-a, not sufficient to offset the effects of inf lation--"the real gain in 
school expenditures suffered a setback for the first time in many years" 
(Sc hool Management > 1972:21). Schools are indeed feeling the pinch. 
But the cost-effectiveness of CAI is contingent upon projections into the 
future of the decreasing cost of CAI as it becomes more widely utilized, 
which is a fair risk for individual school districts to take at early 
stages of development. But technology companies would need assurances 
of a sizable market before developing the software. 



He suggests a not too different case can be made for the problem of ITV* 
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It would seem that in order for school districts to benefit from 
the potential cost-effectiveness of CAI, they would have to jointly 
decide to invest in it or some form of centralized decision making would 
have to take place. Given PPBS projection costs into the future, school 
districts might well make joint decisions to inve.a in CAI (perhaps 
encouraged by state agencies) --not such a far-fetched notion, considering 
how many school districts independently decide to use the same textbooks. 
It seems not such a big step to become aware of the fact that they so 
often end up using the same materials, anyway. 

Berkman suggests that school districts might initially invest in 
computers for the purposes of CMI and that, once the schools had r.he 
hardware, CAI software would more likely follow. By whatever route, it 
is difficult to imagine schools in a money pinch, utilizing PPBS, not 
ultimately turning to the relatively cost-effective educational technology. 
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VI. YES, BUT. . . 

The possibility of large-scale use of educational technology in the 
public schools is much favored by the growth of the accountability move- 
ment; the concomitant introduction of PPBS into education; the possibility 
of PPBS demonstrating existing and potential cost-effectiveness of tech- 
nologies; and the use of incentives, particularly by state and local 
governments. We should, however, consider the possibility of grave 
consequences, if these are the major conditions under which the technology 
is y.o be utilized, not the least of which is the behaviorist learning 
theory upon which so much of the technology is based. 

Behaviorist learning theory is based on the notion that man is^ his 
behavior and that his behavior is totally shaped by the influences in his 
environment, by the reinforcements it offers. By controlling that enyiron- 
rnent, by positively reinforcing those behaviors we choose to see continued, 
we can totally control that behavior. Man has no needs that are not con- 
trolled by his environment. Mis consciousness of a "need" for freedom 
is generated by negative reinforcement. He does not resist control per se, 
merely avarsive control. If we could improve our analyses of behavior 
and appropriately apply positive reinforcements, eliminating negative 
reinforcement, we would eliminate his resistance and this supposed "need" 
fo:: freedom we have come to value. (Skinner, 1971) 

As there is clearly no consensus on behaviorist theory as fact, we 
might do well to consider other theories on the nature of man. George 
Herbert Mead's (1934) doctrine of emergence posits mind and consciousness 
emerging in the process of social interaction. But there are two dimen- 
sions of self. The biologic "I" is generic to the organism (even without 
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consciousness) . It is the impulsive, active, creative dimension of 

self — which offers resistance to the "me" which is the socialized, 

conventional self, shaped by reinforcement. There is more to man than 

his behavior. Skinner's man, despite his aversion to negative controls, 

is essentially a passive, socialized "me" whose behavior is totally 

controlled. Mead does not ignore reinforcement and controls; they are 

necessary for the self to emerge. But within the self there is always 

a t3nsion between the socialized "me" and the authentic "I". The biologic 

"I" implies the possibility of repression, resulting from the control of the 

"I" by the "me". Skinner's man might resist negative controls, but he 

does not resist controls per se . Subjected to the appropriate schedule 

of reinforcements, without aversive control, he cannot be repressed. 

Freud's emphasis on the biological and psychological nature of man 

brings into relief thv question of repression. Man lives his life by the 

pleasure principle; he seeks pleasure and avoids pain. He experiences 

pleasure when his human needs are met. Some of his needs are physical, 

but these cannot be satisfied completely. Their satisfaction is limited 

by his own body, by the forces of nature, and often by civilization (e.g. 

class structure). But not all his needs are physical. 

The desire for freedom that makes itself felt in a human community 
may . . . have its origin in the primitive roots of the personality, 
still unfettered by civilizing influences, and so become a source 
of antagonism to culture. (Freud, 1929:780) 

Man has a need for freedom, but he also has a need for security. 

"Civilized man has exchanged some part of his chances of happiness for a 

measure of security" (Freud, 1929:788). His needs extend far beyond those 

allotted Skinner's man, shaped by controls, which would not exist if our 

culture chose to properly reinforce him. Indeed, those very controls take 

their toll. 
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. . .it is impossible to ignore the . . .degree to which the ^ . 
existence of civilization presupposes the non-gratification 
(suppression, repression, or something else?) of powerful 
instinctual urgencies. This cultural privation dominates 
the whole field of social relations between human beings . . . 
(Freud, 1929:781) 

But men do not assively submit to repression. As there is always a 

tension between Mead's ''I'* and "me'', so there is always tension between 

Freud's libido and ego (and/or conscience) --between man and civilization. 

It is ill his nature to resist . 

The goal towards which the pleasure-principle impels us- -of 
becoming happy- -is not attainable; yet we may not--nay, cannot 
--give up the effort to come near to realization of it . . . • 

It does not seem as if man could be brought by any sort ox 
influence to change his nature into that of ants; he will 
always, one imagines, defend his claim to individual freedom 
against the will of the multitude. (Freud, 1929:775;780-1). 

It is not a totally fruitless struggle. Freud points out that "we cannot 

abolish all suffering, yet a great deal of it we can, and can mitigate 

more" (Freud, 1929:776). And Marcuse's (1955) reinterpretation of Freud 

makes the distinction between repression and surplus repression. Repression 

will always be with us because of man's constitutional limitations. But 

surr)lus repression is suffered over and above what we can attribute to man's 

nature. It is imposed upon man by the social structure, by the institutional 

arrangements in that structure, by the distribution and exercise of power 

in society. These are subject to change by man. There is hope, for 

there is always resistance. 

For Karl Marx (1956; 1864), as for Mead, mind and consciousness are 

not given; they emerge within a social context. Social being precedes 

and determines social consciousness. But Marx also sees man as a natural 

being with human needs. Over and above the animal needs, he has a need 

to express and fulfill his active, creative potential. He proves himself 
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a conscious species-being by creating an objective world, which he does 
even when freed of inunediate needs. And it is only then that he is 
truly free, acting on human needs . Insofar as social structure stifles 
man's active nature (Mead's "I"), man is made less human; he is debased 
when he must produce only to satisfy physical needs. 

For Marx, as for Mead and Freud^ the psyche and soma are a unity 
witn biologic human needs. But Skinner's pure behaviorist perspective 
views all psyche and soma determined by controls. What Skinner must 
ultimately girant, while he insists upon the influence of the environment 
on man's very nature — through the process of natural selection, is that 
what may very well have been selected is the need for freedom he so much 
wants to deny. Without the concept of such needs, one cannot appreciate 
the tension between the "I" and the "me'*, between man and social structure. 
Mead, Freud, and Marx approach this tension in different ways, from dif- 
ferent perspectives, but they never lose sight of the active, creative 
dimension of self — of the need for freedom. Skinner's ideal of shaping 
behavior allows for the creation of an infinite variety of "me's", but it 
does not permit an "I" to emerge. It overlooks the question of psychological 
repression resulting from control. 

One cannot ignore vital needs, which are fulfilled or repressed by 
social behavior. Without the concept of such needs, one cannot say that 
a .social system is repressive. One makes of man a passive creature; he is 
an active, creative being. And freedom is not merely a value — it is a 
vital need. 

The possibility of repression of a human need for freedom through 
the large-scale use of a more effective educational technology is related 
tc another important question. If we are to shape behavior by means of 
^ controls, and perhaps ultimately have the means at hand to make of man what 

ERIC 
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what v/e will--what will we make of him? What behaviors will we choose 
to enforce? Skinner maintains that a culture teaches that behavior 
which is necessary for its survival. Likewise, Cogswell »s defense of 
his surveillance and detection system of computer-managed instruction is 
based on the notion that a culture acts in its own interest. But what is 
in the interest of our culture? And what does it mean to talk of the 
survival of a culture? The use of ''culture'* in the abstract suggests an 
artificial unity to the people of our society. 

When we get down to real people we are faced with the fact that a 
''culture'' does not choose behavioral objectives and a "culture" does not 
have interests. People make choices and people have interests. To assume 
that all people in our society have the same interests is to ignore the 
fact that there are innumerable controversies over what to teach in our 
schools: Will we or will we not teach religion? Will we or will we not 
teach about the theory of evolution? sex education? communism? 
To as.'^ume that all the people in our society have the same interests 
means to decide for them what is in their interest (when indeed they 
may well protest). And who will make that decision? 

Beyond the question of what behavioral objectives are chosen, we 

must ask wlio is doing the programming? Skinner suggests that 

There are hopeful signs that the epistemological implications will 
induce experts to help in composing programs. The expert may be 
interested for another reason. We can scarcely ask a topflight 
mathematician to write a primer in second-grade arithmetic if it 
is to be used by the average teacher in the average classroom. 
But a carefully controlled machine presentation and the resulting 
immediacy of contact between programmer and student offer a very 
different prospect, which may be enough to induce those who know 
most about the subject to give some thought to the nature of 
arithmetical behavior and to the various forms in which such 
behavior should be set up and tested. (Skinner, 1968b: SO) 

ERLC 
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The epistemological implications to which Skinner refers are those 
relating to the ability to shape behavior through programmingo But there 
are other epistemological implications in the notion of having ''experts" 
program instruction which might ultimately be delivered to large numbers 
of children as a result of centralized decision making. Shall we have 
"experts" teach religion? evolution? sex education? communism? 
How shall we determine who the "experts" are? Will we really agree? 
To 2rant "experts" access to the minds of our children ignores the real 
epistemological question of how we know what we "know"--what is fact, 
what is truth. This is of particular importance when "knowledge" is being 
comiTiunicated by a thing like a computer and the mystique of the machine 
can lend credence to "facts" derived from a weak knowledge base. 

The dangers of ruling out epistemological questions and the mystique, 
of the machine apply as well to systems like PPBS that employ analysts to 
feed data into a computer that spits out cost -effectiveness analyses of 
alternative courses of action. PPBS is a model of a system consisting 
of a number off interdependent variables which are subject to manipulation 
tc achieve desired goals. Implicit in the definition of that system is the 
existence of an environment in which that system operates--including all 
elements outside that system, over which we have no control and acc'ept as 
given . The environment is treated somewhat as we might treat the law of 
gravity: no one is responsible for it, we cannot do anything about it, 
we must submit to it. (See Boguslaw, i965 and Levit, 1972) At this point, 
it becomes important to ask who is defining this system and determining what 
variables are given and beyond anyone's responsibility. It may be in the 
interest of school personnel to take the low socio-economic status of 



their students as a given and beyond responsibility. It may not 
be in the interest of the students to treat their lot so. 

Similar questions arise when measures of cost-effectiveness are 
derived from numbers somehow attached to these variables. What does it 
meaii to determine that program A produces 60 students at a high level of 
achievement, program B produces 90 students at a lower achievement level, 
and program C produces 150 students at an even lower level, all at the 
same cost? (Carpenter, 1971a) Who determined what alternatives would 
be evaluated (and what alternatives not considered)? Who is to choose 
which alternative will be implemented? 

There is at least one other issue which cannot be overlooked. When 
a program of instruction is found to be "better" by means of a cost- 
effectiveness analysis which very likely takes as a given and holds 
constant. the life-conditions of children, and that program is mandated, 
we may very likely be inhibiting the innovation of alternative programs 
which might improve those life-conditions. 

The development of new educational technology may or may not prove to 
be more effective or less costly than other programs of instruction. In 
any event, it is incumbent upon us to urge that, in the determination of 
how those programs will be used, every effort is made 1) to assure and 
encourage every possible avenue of access to decision making — for producers 
and consumers and anyone else who is interested, and 2) to assure that 
no program be mandated. Only then can we reap th6 possible benefits of 
now technologies without blighting the harvest. 
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VII. SUMMARY 

The convergence of three major factors in our society today 
constitute a powerful force conducive to the large-scale utilization 
of technology in education: 

1) The very fact of the development of impressive technologies 
in the DoD, NASA, and AEC, and their interest in finding and 
encouraging civilian application of their discoveries; the use 
of systems analysis and PPBS, responsible for those discoveries, 
as a model for the solution of social problems, including edu- 
cation; the notion of incentives to encourage the use of 
systems analysis. 

2) The current predicament of local school districts, pressured by 
calls from the public and from state and federal authorities 
for pedagogical and fiscal accountability- -encouraging the 
receptivity of educators to the idea of utilizing available 
technology, systems analysis, PPBS, and incentives to provide 
better education. 

3) A behaviorist learning theory which supports, even demands, 
the systematic use of technology to improve education. 

In sum, we have seen educators turning to technology in an attempt to 
solve fiscal and pedagogical problems—all the while groping toward a 
refined application of PPBS cost-effectiveness analyses which will 
ultimately determine, probably favorably, the future fate of tcchnology- 
in educatjop.. 
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Moreover, we have cautioned against pitfalls in applying systems 
analysis, noted for its success in improving nian-machine systems, to 
educational problems — taking care not to place so much emphasis on the 
aspects of quantification and machinery that we lose sight of the nature 
of man. We urge that the careful consideration of ultimate goals not 
be neglected, that the consequences of method no^, be overlooked, so 
thax our work not be in vain. To insure against such an eventuality, 
we must assure that all access to decisionmaking remain ever open. 
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APPENDIX A 



VOUCHERS, FREE SCHOOLS, AND COMMUNITY CONTROL 



Some critics, comparing schools to other organizations, such as 
business enterprises, find the problems of schools rooted in their 
monopoly character. Mark Hanson's (1971) views, representative of this 
analysis, may be summarized as follows: 

Most organizations must compete for scarce resources (input) and 
for markets for their products (output). In order to survive, they 
must adapt their product (goals) and productive processes (means) to 
changes in the availability of resources and demands for products. 
Having selected their goals, they engage in a sequence of planning 
processes such as that defined by the U. S. Bureau of the Budget: 

1. defining objectives 

2. measuring anticipated output of alternative programs which 
might be adopted to achieve the stated objectives 
determining the total system costs of alternative programs 
on a multi-year basis 

4. selecting the most appropriate of the alternatives 

5. providing a systematic flow of information on outputs 
and costs for periodic evaluation of progress (Hanson, 
1971:19) 

The planning sequence is dependent upon selecting operational goals-- 
goals which admit of a means of testing their relationship to alternative 
processes designed to meet them. 

But, says Hanson, schools usually define their goals in non-operational 
terms, such as: 

1. intellectual discipline 

2. citizenship and civic responsibility 

3. economic interdependence and vocational opportunity 

4. social development and human relationships 

5. moral and ethical character 

6. self-realization (Hanson, 1971:19) 



-67- 



Such goals are so broadly defined that no measuring tools are available 
to evaluate the degree to which they are reached or the relative effective- 
ness of alternative programs. The reason schools manage to survive with- 
out selecting goals according to (market) demand or evaluating the degree 
to which and the processes by which they are met is that the schools 
maintain an effective monopoly of the education enterprise. They need not 
compete for resources or markets. Students must attend them and accept 
whai:ever program is provided. Funding is dependent upon neither school 
nor student performance; it is assured through public funding. 
(Hanson, 1971) 

Vouchers 

The accountability movement and taxpayer resistance are placing 
strains on schools that require a change. Some who view the source of 
trouble in the school monopoly maintain that the best way to foster the 
necessary changes is by forcing the schools to compete for money and 
students. A number of proposals have been made to fund education by 
providing parents with educational cash vouchers redeemable by schools 
in which the parents choose to enroll their children. 

The leading advocate of educational vouchers, Christopher Jencks, 

promotes his plan in a study funded by the U. S. Office of Economic 

Opportunity on a variety of voucher proposals. A summary of Jencks' plan 

includes the following provisions: 

— Parents choose between competing schools. 
— Schools must be open to all applicants. 

— Schools must accept the voucher as full payment for the cost 

of a child's education. Parents may not add money to the value 

of a child's voucher. 
— Each school must make information available which will enable 

parents 'to make wise decisions. 
— A new independent agency, the Education Voucher Agency (EVA) 

enforces these regulations and administers the voucher program. 
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--The value of the voucher is supplemented for the poor, to 
enable schools to develop special programs for these children. 

--All kinds of schools are included — private and religious schools 
as well as public schools. 

(Mecklenburger, 1972:23; see also, Jencks, 1970) 

Plans by OEO to implement a voucher experiment based on Jencks' plan 
have been hampered by the following objections; that educators relinquish 
to parents the choice of school programs; that parents could add no money 
to the voucher; that EVA's might displace school boards; that private and 
parochial schools would be supported by public monies; that teacher rights 
and desegregation might be adversely affected. The experiment is underway 
now at only one cite, Alum Rock School District of San Jose, California. 
But the Jencks plan has been compromised in that; only public schools 
are included; only one-third of the school district is participating; 
the Alum Rock Board of Education maintains control, advised by an Edu- 
cation Voucher Advisory Committee of parents; the choice is not among 
schools but among prograjiis. This is a one year demonstration, the 
continuation of which is dependent upon three conditions: that the 
California legislature pass a bill authorizing the creation of an EVA 
and the use of public monies for private and parochial schools; that 
Alrjn Rock doubles the number of experimental students in 1973; that OEO 
has. the funds to continue the program for the next five to seven years. 
(Mecklenburger, 1972) 

The results of the Alum Rock demonstration and any further voucher 
experiments remain to be seen, including the possibility that they might 
foster the kind of systems analysis that Hanson and others find necessary 
to the solution of school problems. We can look forward to a report from 
the Rand Corporation which has contracted with OEO to evaluate the ex- 
periment. 
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Free Schools 

The **free school" movement is made up of people concerned with the 
direct effect of the public school monopoly on the lives of children. 
They perceive a rigid, authoritarian bureaucratic structure which stifles 
the natural urge to learn and create with its prescribed curricula and 
methocs, stressing discipline and grades and credentials. They perceive 
a structure which creates an artificial environment, distorting the 
nature of man and learning and reality--at best, irrelevant to the needs 
of children; at worst, inhumanly destructive. To save children from the 
boredom, frustration, feelings of impotence and dependency generated by 
such a structure, they have established hundreds of their own independent 
"free schools," variously designed to allow children to grow and develop 
and pursue their own interests unhampered by prescribed, regulated 
direction. It is hoped that, freed from the fetters imposed in the public 
schools, children will develop a sense of their own integrity and of 
others* and the capacity to relate to each other iii the real world — so 
as to create the conditions needed to live human lives. 

Although the movement is hardly unified, philosophically or in 
practice, the rejection of the strictures of public schools (and often 
of the entire social structure) in many cases leads to a near total 
non- structured, non-academic learning experience. Exceptions can be 
found in some inner- city free schools for the minority poor, such as 
those advocated by Jonathan Kozol, which do stress the teaching of 
academic skills deemed necessary for these children to better their condition. 

Most free schools have a rather tenuous existence due primarily to their 
acute lack of funds, teachers working for little or no pay, and the trials 
and tribulations of putting their ideals into practice. (See: Stretch, 1972 
Marin, 1972; Daniels, 1972; Greenway, 1972.) 
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Coimnunity Control 

Partly because of the ephemeral nature of free schools, some inner- 
city residents, agrieved by the experience of their children in public 
schools, have begun to demand "community control." Levin (1972) presents 
the case for decentralized conununity control of public schools, summarized 
as follows: 

Large centralized school districts have been rigid and unresponsive 
to the needs of the minority poor. They have failed to provide these 
children with the formal education necessary to improve their life- 
conditions. Curricula and methods designed for white middle class 
children have dealt destruction to the growth and self-concept of poor 
black children, leaving them with feelings of incompetence and impotence. 
Moreover, proposed solutions, such as integration and compensatory edu- 
cation, are an affront to the dignity of the black man. The notion of 
integration has not only meant false promises but, indeed, implies that 
blacks can only learn in the company of whites. Compensatory education 
hai; meant little more than bigger and "better" doses of the same debili- 
ta'cing white middle class approaches. The view is that only by over- 
coming their own sense of impotence and gaining control over their 
schools can the black community assure that the cultural uniqueness of 
their children will be appreciated and their needs met. (Levin, 1972) 

Efforts to gain community control have been hampered by strong 
opposition from centralized school administrations and teachers' unions 
unwilling to relinquish control of education (and their jobs) to dis- 
gruntled laymen. To date, the community control movement has had little, 
5.f any, success. 
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# # # 

Though the education voucher, free school, and community control 
movements are certainly parts of the larger accountability movement, 
they are somewhat on the periphery insofar as they are less directly 
concerned with systematically relating input to output. And although 
the argument has been made that vouchers would force schools to do so, 
these three developments concentrate primarily on the selection of 
obj 'Actives and quality of output. Their focus is on breaking the 
monopoly of large public school systems in the area of decisionmaking 
and control over the lives of children. They are based on the notion 
that insofar as this is done, the quality of the school experience and/or 
its output will improve. On this periphery, the challenge has gone beyond 
holding schoolmen responsible for doing a job well to doubting it can be 
done~-within the confines of the current structure of school bureaucracy. 
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APPENDIX B 

TESTING STUDENTS AND DETERMINING ACCOUNTABILITY 

While increased interest in individualized instruction has been 
a response to calls for accountability, the determination of pedagogical 
accountability ultimately rests on the measures employed to assess output 
(learning or change in pupil behavior). And although there are a few 
cries on the periphery that there should be no testing per se_, the debate 
actually comes down to "how to measure," not whether or what to measure: 
If educators are to be held accountable, then output must be measured. 
We measure changes in behavior in the areas we have intended to influence. 
We can then attempt to determine what influences, in fact, effected those 
changes. 

Testing Students 

The plethora of literature on testing for accountability focuses 
on the debate over the relative merits of criterion-referenced tests 
and norm-referenced tests. The examples of individualized instruction 
discussed above indicated the importance placed on the establishment 
of measurable behavioral objectives and the determination of criterion 
pe:rformances for meeting those objectives. But most public schools in 
this country, including many attempting to individualize instruction, 
employ norm- referenced tests. Jason Millman (1970), editor of the 
Jo urnal of Educational Measurement, has presented the case for criterion- 
referenced tests which essentially makes the following argument: 

Norm-referenced tests are designed to distinguish the individual's 
performance from that of others on the same testing device. They are 
comparative measures wherein a norm is established for the group being 
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tested. The students are essentially competing against each other; the 

individual's score is either average, above average, or below average. 

Most standardized achievement and IQ tests, as well as teacher-made tests 

on which students are rated A, B, C, etc., are norm-referenced tests. 

By definition, norm-referenced tests are designed to assure something 

of a normal distribution of scores; that is, some students will always 

fall below average. 

By contrast, criterion-referenced tests measure an individual's 

performance with reference to some established criterion, or performance 

standard, related to a specified behavioral objective. The student's 

score does not compare his performance to that of others; rather, it is 

a measure of the student's own progress toward mastery of any given 

objective. The use of norm-referenced tests is inappropriate when the 

primary concern is what an individual student has learned. 

A key task of our schools is to maximize the amount of 
a subject that each student has "mastered." Indeed, a reason 
for having individualized instruction is to maximize achieve- . 
ment by appropriate pacing and provision of instructional 
materials. The rational management of such an individualized 
instructional system requires knowing whether each student can 
perform at some criterion level on measures of the component 
objectives of the system. (Millman, 1970:227) 

Norm-referenced tests aside, there are those who are skeptical of 
the notion of structuring teaching around measurable performance objectives, 
maintaining that there are higher mental processes, such as insight, crea- 
tivity, an inquiring mind, which constitute goals that cannot be defined 
in terms of quantifiable short-range behavior objectives, that we ought 
not attempt to do so, nor risk the possibility of omitting them for lack 
of such a measure. This is a valid point to raise. Theoretically, 
of course, such behavioral objectives need not be quantifiable, although-- 
to be sure--a good part of their advocates do seem to be making that demand. 
Nonetheless, performance criteria can be specified for these objectives which 
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need not be quantified. Still, it should be noted that, however much 
today's schools include insight and creativity among their stated 
goals, it would be a rare school indeed which determined its own success 
and passed and failed students on the basis of such variables. This is 
not for nothing. For the life-chances of today's youth, their chances for 
success in our society, are largely determined by their ability to compute 
numoers and to speak and read and write the English language, . It is 
precisely in such basic skill areas that most standardized norm-referenced 
tests measure achievement and efforts to construct criterion- referenced 
measures are centered. 

Of course, it is possible to specify standardized test scores as 
criterion performances. Most achievement tests were standardized some 
years ago; they are not restandardized every year. Hence, although 
at the time of standardization, some students' scores (by definition) 
fell below the norm, one could specify that subsequent populations 
tested on these measures must demonstrate proficiency at the level of 
that norm to achieve mastery of a given subject matter. It is the use 
of standardized test scores as cx'iterion performances, notably by schools 
employing performance contractors, that has generated the greatest con- 
troversy among those concerned with testing for accountability. 

Lennon (1971) and Stake (1971) expound most of the arguments against 
the use of standard scores as criterion performances, based primarily on 
the fact that they were not intended as such--that criterion-referenced 
tests and norm- referenced tests are constructed for different purposes 
and that the exigencies of testing are such that neither test is an appro- 
priate measure of the other's purpose; Criterion-referenced tests are 



* 

See Chapter IV. 
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constructed so that each item bears a one-to-one relationship to speci- 
fied behavioral objectives, measuring competence in the performance of 
specific tasks. They are intended to measure learning. It is not their 
function to be predictive of future performance on other specific tasks 
or those of a more generalized nature — nor are they. By contrast, 
sta:idardized achievement tests are constructed to measure correlates of 
learning, not actual learning itself. The items on these tests are not 
related to specific behavioral objectives; they are intended to get at 
a much wider range of content, sampling the universe of what has been 
taught. Standardized' achievement tests are not intended to measure what 
specific task an individu-itl student is now capable of performing. Their 
value rests in their relatively high predictive validity, indicating 
proDable future performance. 

It is on the basis of this difference in purpose and construction 
that charges of "teaching to the test** have been leveled against some 
performance contracts (Wardrop, 1971; Lennon, 1971; Stake, 1971). Since 
criterion-referenced tests are intended as direct measure of learning, it 
is appropriate to teach precisely those behaviors assessed by each test 
item. But performance contractors are generally evaluated (and paid) on 
the basis of student performance on standardized achievement tests. Under 
those circumstances, to teach those precise behaviors which will be assessed 
by specific test items that are intended as samples of a universe of be- 
haviors---without reference to that universe — invalidates the test; the 
meaning of the scores is distorted; scores no longer have their intended 
predictive validity. 

Those who would teach to samples of behavior as though they were 
merely behavioral objectives are further discredited by the fact that when 
achievement tests are standardized, raw scores (usually the number of items 
right) are translated into grade equivalents, indicating the average grade 
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placement of all students with a given raw score. Most performance 

contractors are paid on the basis of their ability to raise students • grade 

equivalents, usually by one year. But "the average annual ^growth* on 

most standardized tests is only a few raw-score points" (Stake, 1971:586; 

see also, Lennon, 1971 and Wardrop, 1971). 

Another major objection to the use of standardized tests to measure 

learning relates to the aforementioned fact that these tests are designed 

to predict long-range performance. But again most performance contractors 

are paid on the basis of short-range gains in the scores of individual 

students. A rise in grade equivalent is translated into a gain score-- 

which is highly unreliable as a measure of individual improvement. 

The error of measurement of a gain score may very easily equal 
or exceed the amount of gain normally to be achieved in a short- 
term intervention. . . 

It is almost instinctive to react to this state of affairs 
by saying "Well, let compensation be based on average gain for 
a group and avoid the messy question of unreliability of individ- 
ual gain scores." Clearly the intent of the performance con- 
tract is to foster the academic growth of every participating . 
learner; and no evaluation plan will be acceptable that allows 
failure by a significant fraction of the group to make good gains 
to be offset, in calculating payment, by better- than- average gains 
by others. (Lennon, 1971:10) 

Leniion and Stake discuss a number of ways in which errors in individual 

gain scores might be reduced, Lennon preferring to lengthen the period 

over which changes in buhavior is assessed. Stake being more optimistic 

about the possibility of measuring short-term changes--but he cautions: 

"Corrections for the unreliability of gain scores are possible, but they 

are not likely to be considered if the educators and contractors are 

statistically naive" Stake, 1971:587). 

Lennon, Stake and others (e. g. Saretsky, 1972) go on to raise 

additional questions with respect to the use of standardized achievement 

tests to measure learning. Suffice it to say that any school which would 
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do so must come to terms with these issues--for, of course, they apply 
to- such use by any school. The arguments are currently directed at their 
use by performance contractors because of the high visibility of con- 
tractors in their bold attempt to make schools accountable by providing 
guarantees and because they are paid on the basis of gain scores. 

Indeed, as one examines the literature encompassing the testing 
controversy, one cannot help sensing that much of the furor centers on 
determining the accountability of educators for the performances they 
effect--and rewarding them accordingly. For the arguments of critics 
notwithstanding, schools have been using standardized achievement for 
many years to measure the learning of students (given credence by the 
tests' predictive validity). It is on the basis of such tests that 
"low-achievers*' and "slow-learners" are so labeled. And it is on 
the basis of these labels that the dismal life-chances of many of these 
children are determined. The notion of judging the efficacy of school- 
men--and paying them--on the basis of these same tests has generated the 
closest scrutiny of the tests--and performance contractors. This is 
perhaps 4iiost clearly illustrated by the controversy over the use of 
extrinsic rewards by performance contractors. 

Amid loud protest against the deleterious effects of extrinsic 
rewards, Tuinman et al. (1972) demonstrated that contractors might appre- 
ciably raise scores on achievement tests merely by increasing motivation 
with the promise of rewards made just prior to testing. Hence, contractors 
mifht collect sizable sums of money without teaching, i.e. without effecting 
learning. Aside from the fact that that is precisely what many teachers 
have been doing without so much as raising achievement test scores (part 
of the reason for the accountability movement), Tuinman et al^. ,*s findings 




and their high-achieving peers promoted and rewarded. 
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might better be interpreted as evidence of insufficient motivation 
for many students to perform without extrinsic rewards. Apparently, 
the "standard levels of motivation" available to students not promised 
rewards are inadequate to get students to tell us what they know. It 
seems a bit peculiar .that when we can improve student performance — merely 
by increasing raotivation--those same tests that labeled those same students 
"low-achievers" are not called into question, especially given the grave 
consequences for these students and the focus of the accountability 
movement on improving their performance (Anderson and Lipman, 1972a and 
1972b; see also, Saretsky, 1972.) Ironically, while the doctor is re- 
examining his instruments in preparation for a new treatment, he would 
suspend the old, and the patient gets caught in the middle. For until 
he gets those instruments calibrated, he would impose a double standard. 
The tests are good enough to get you into the "dumb class" but not good 
enough to get you out. 

This is not to discount the validity of the many questions raised 
in the context of efforts to make schools accountable and to develop 
appropriate measurement techniques. It is to recognize that the task 
is no simple one, that efforts to work it out are in an embryonic stage, 
and that it may take some time. Hopefully, in the interim, no double 
standard need apply; and hopefully, in the process, the need for suffi- 
ciently motivating students will not be overlooked. 

as described by Tuinman et, al_. (1972:216) 

In the context of the continuing debate over the pros and cons of 
criterion vs. norm-referenced tests, we might note that while Millman 
maintains that criterion-referenced tests can be readily constructed 
(and we have noted above progrjams of individualized instruction pre- 
paring their own), Lennon and Stake remain skeptical of the reliability 
and validity of these tests, indicating that the construction of quality 
criterion-referenced tests is no small task. 
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Determining Accountability 
While the techniques of measuring learning are being refined, 
efforts proceed to develop methods of assessing school accountability. 
In order to deal with the problem of how to set criterion performances 
for schools, these accountability measures are relative, involving com- 
parisons among schools and/or groups of students. Also;, since relatively 
few schools as yet attempt to specify criterion performances differenti- 
ally for students and since efforts to assess school accountability 

★ 

might be rather more complex in these cases, initial proposals assume 
a uniform set of criterion performances for all students. 

The Yardstick Project is a rather modest effort currently being 
tested in twenty-four Ohio school districts. As suggested above, at 
this early stage of the accountability movement, many of the questions 
concerning the measurement of learning are raised in the cor/text of 
using these measures to determine the output of educators~fcr which 
they are held accountable and on the basis of which they are appropriately 
compensated. Hence, Lennon's objection to the notion of performance con- 
tractors' being paid on the basis of average gain which might detract 
from the concern for individual growth. Since the Yardstick Project 
is less immediately concerned with the matter of providing incentives 
for educators and is intended first as a tool for school district 

•k-kH 

administrators to compare the output of schools, it does rely on 

claims of fiscal and pedagogical accountability by such programs of 
individualized instruction > notwithstanding 

though teaclier pay does fall within the realm of its concerns 

*** though it ^uld be logically adapted for use within individual schools 
and classrooms, "^^and need not preclude concern for individual student 
growth 
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average gain scores. In fact, the Yardstick Project might best be viewed 
as a representation of what one might expect from first efforts to insti- 
tute '^accountability** in the midst of debate over its meaning and methods 
without actually taking into account many of the issues being raised. 
Nonetheless, it is as least a first effort. 

The first task the Yardstick Project set for itself was to find a 
means of evaluating the comparative performance of schools within a local 
district which would not be distorted by the differences in student bodies, 
so often mentioned as factors precluding comparison among schools. 

The hypo th<^sis : Differences in the achievement growth of a group 
of pupils in one school, compared to the growth of a similar group 
of pupils in another school, depends less on the backgrounds of the 
pupils than on the programs offered in the schools themselves. 

( School Management /^ 1970:20) 

Standardized achievement tests were chosen as "universal yardsticks"-- 

first, because scores on them are available in the records of most schools; 

and second, because they embody norms for each grade level. Hence,' 

average gain scores could be used to determine each school's "yearly 

value added to pupils' scores." To make certain that "value added" scores 

were comparable, Yardstick researchers utilized a computer to isolate 

"conditions" of learning which might differentiate among groups of students. 

From among hundreds of variables, they found the students' IQ's and their 

fa'^hers' occupations to be the best indicators of such differences, 

Workir,, within these indicators, they verified that achievement levels 
are cl^ ^.sely related to intelligence and background. But they also 
found that achievement growth is not significantly dependent on 
those factors. It is more dependent on the students' experiences 
in school. ( School Management , 1970:20) 



-81- 



Now the Yardstick Project coulc} develop its ^'growth gauge," a series 
of charts for the district as a whole and for each school. The former 
indicate grade-by-grade changes in achievement as students move through 
the system, as well as yearly changes in student body make-up, in terms 
of IQ and socio-economic background. The latter indicate each school '5 
yearly value added to students' scores by IQ and fathers' occupartion. 
( School Management , 1970:20-21) 

Having developed the growth gauge to pinpoint which schools con- 
tribute how much yearly value added to which students — "an instrument 
'^for measuring the output of past and present school district policies 
and programs," the Yardstick researchers set about their second task: 
a planning model to evaluate the implications of proposed changes in 
policies and programs. The planning model utilizes the computer to 
process "data available in most school files" for cost-benefit analysis. 

The model will project the consequences of juggling class 
sizes, raising the teacher pay scale, building a new school, 
or performing a host of other altematives . The advantage, 
of course, is that the effects of alternate plans can be 
analyzed without actually trying them, without committing 
^ the district and without spending a great deal of time or 
money. ( School Management , 1970:21) 

While the Yardstick Project may not deal with all the complexities 
the issues raised in the context of the accountability movement, it is 
putting "accountability" into practice, insofar as it is instituting the 
utilization of measures of school output and tools to aid pedagogical 
and fiscal accountability. 

A rather more sophisticated approach to pedagogical accountability 
has been developed for New York City schools by the Educational Testing 
Service in collaboration with a committee comprised of representatives 
of the city's main educational power groups. While the plan has yet to 
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be publicly released, a preliminary proposal set forth by Henry Dyer 

(1970), of ETS, outlines a plan with several unique features. Like the 

Yardstick Project it assumes the joint accountability of the school 

staff. But Dyer emphasizes that accountability is a two-way street: 

the professional staff is to be held accountable 

for knowing as much as it can (a) about the intellectual 
and personal-social development of the pupils in its 
charge and (h) about the conditions and educational services 
that may be facilitating or impeding the pupils' development. . . . 
for using this knowledge as best it can to maximize the de- 
velopment of its pupils toward certain clearly defined and 
agreed-upon pupil performance objectives. 

[But the ] board of education has a corresponding responsibility 
to provide the means and technical assistance whereby the staff 
of each school can acquire, interpret, and use the information 
necessary for carrying out the two foregoing functions. 

(Dyer, 1970:206) 

Most of the literature on accountability concentrates on the need 

for relating input to output--that is, relating material and human 

resources expended to the quality of pupil performance that accrues. 

But Dyer explicitly recognizes four distinct groups of variables in 

his pupil-change model, stressing that any measure of accountability is 

meaningless unless all four are taken into consideration: 

The input to any school at any given level consists of the 
characteristics of the pupils as they enter that level of their 
schooling: their health and physical condition, their skills in 
the three R's, their feelings about themselves and others, their 
aspirations, and so on. The output of any school consists of 
the same characteristics of the pupils as they emerge from that 
particular phase of their schooling some years later. 

(Dyer, 1970:207) 



The characteristics herein considered are those for which the 
schools are to be held accountable, those for which its activities arc 
designed to effect a change. Dyer's redefinition of input stresses the 
fact that pupil characteristics are not the same as they enter the school 
and that they are a fixed condition over which the schools have no prior 
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contrci. But they are not comparable to the Yardstick Project's use of 
IQ as a '^conditioner of learning." To the contrary, it is unlikely that 
IQ would be among the pupil characteristics considered, as few schools 
would include it among the characteristics they intend to change. 
(Dyer also stresses that these characteristics would not be limited to 
acaiiemic areas.) That output measures would be applied "some years later" 
indicates that Dyer adheres to the notion (discussed above) that accounta- 
bility is not possible for short periods of time. But these output measures 
are not themselves measures of accountability. To Dyer they are meaning- 
les,s without taking into account a third group of variables--the sur- 
rounding conditions within which a school operates and which might influence 
teaching and learning. This group of variables encompasses home conditions, 
including the physical condition of the home, parents levels of income and 
education, family pressures; community conditions, including ethnic charac- 
ter, population density, available social agencies, degree of industriali- 
zation, etc.; and school conditions, including school plant, pupil- 
teacher ratio, esprit de corps of the staff, etc. Dyer points out the 
need to distinguish those surrounding conditions which would be hard for 
the school staff to change from those which would be easy to change, 
suggesting that the staff would not be able to influe^c^^parents ' socio- 
economic position but might well be able to ^.^t&l^^ie their attitudes 
toward the school by involving them in its operation, that the school plant 
might be difficult to change hut the staff's esprit de corps might not. 

The identification of hard-to-change as contrasted with easy- 
to -change surroundi^ig conditions is of the utmost importance in 
in working toward objective criteria of proftissional accounta- 
bility, since the staff of the school can hardly be held account- 
able for changing those fac-.ors in its situation over which it has 
little or no control. (Dyer, 1970:207) 

The measure of professional accountability or school effectiveness 

would be derived from measures of Input, output, and hard- to-changc 

i 
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surrounding conditions through a series of regression analyses. It 

would indicate where corrective action needs to be taken. What that 

action might be would be determined by close scrutiny of the easy-to- 

chaiige surrounding conditions and the fourth set of variables — those 

making up the educational process, encompassing all activities in which 

the school engages to effect changes in pupil characteristics. Of the 

latter Dyer ask three questions: *'l) Are they adapted to the individual 

needs of the children in the school? 2) Do they work, that is, do they 

tend to change pupils in desirable ways? 3) What, if any, negative side 

effects may they be having on the growth of the children?" For Dyer, 

it is crucial that all four sets of variables be taken into account. 

If a school staff is to maximize pupil output in any particular 
way, it must be aware of the nature o^f the interactions among the 
variables in the system and be given sufficient information to 
cope with them in its work. This in turn means that, insofar 
as possible, all variables in the system must be measured and 
appropriately interrelated and combined to produce readily in- 
terpretable indices by which the staff can know how much its own 
efforts are producing hoped-for changes in pupils, after making 
due allowances for those variables over which it has little or 
no control. I call such indices school effectiveness indices 
(SEI's). (Dyer M970: 207-8) 

Each SEI indicates the degree to which a school has been effective 
in furthering one area of pupil development over several years. It also 
differentiates among students with high, medium, and low levels of output 
for that area of development within that school, so that the school can 
determine whether it is differentially serving the three groups in each 
area. Which areas of pupil development might be included in a profile 
of SEI's, Dyer stresses, must be determined by all interested groups: 
pupils, parents, teachers, administrators, board members. While different 
schools may not have the same objectives, measures of input and hard-to- 
change surrounding conditions must be applied to all schools in a district 
^ in order to derive appropriate SHPs for any given school. The procedure 

ERIC 
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would work as follows to assess effectiveness in teaching reading in 

the middle grades of an elementary school: Measures of input in reading 

at the beginning of fourth grade and averages of hard-to-change condition 

variables are summarized '*in terms of the grade six predicted average 

reading levels as determined by the regression analysis." Output measures 

of actual reading levels are taken at the end of grade six. 

For each school, the discrepancy between its predicted grade 
six reading level and its actual grade six average reading level 
is used as the measure of the effectiveness with which it has 
been teaching reading over the three year period. It is the 
discrepancy between predicted and actual level of performance 
that is used to determine the SEI in reading for any school. . . . 



.. . .it should be noted that the proposed method of computing 
school effectiveness indices automatically adjusts for the 
differing circumstances in which schools must operate. 

(Dj-er, 1970:209) 

Easy to change conditions and educational process variablei^ of schools 
with comparable predicted levels but different actual levels would be 
systematically analyzed to determine possible corrective action for 
deficient schools. 

Dyer's plan is presented here at length, because it is one of the 
few proposals that takes into account many of the complexities of assess- 
ing school accountability. While Dyer grants that measures of many of the 
variables are not readily available, he notes that many are; and he 
urges that first steps be taken toward eventual full-scale implementation. 
Apparently New York City is about to take those steps--with a 1975 target 
date suggested for field tests to begin, the interim to be spent on the^. ; 
setting of objectives and the development and improvement of measures. 
(Bard, 1972) 



We might note, as well, that in Oyer's choice of method, he has 
explicitly rejected the conventional use of grade cqui v^ilcncics. 

All variables would have to be quarit i f i cd. 
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Project Yardstick and Dyer's proposal are both based on the notion of 
collective accountability of a school's staff, finding out which schools are 
deficient in piwiding education and taking corrective action. Ultimately, 
however, for many people accountability means that the individual is respon- 
sible for doing his job competently. Dyer rejects out of hand the possibility 
of separating the influence of any given teacher from other school variables. 
And the notion of holding any individual responsible for a job not -so-well done 
while attractive to parents and taxpayers, tends to make educators nervous. 

Rand analyst Stephen Barro (1970) has dealt with this problem at length 
and has proposed a technique not unlike Dyer's. Like Dyer, his first concern 
is that no one be held responsible for the influence of variables over which 
he has no control, and Barro finds a multiple regression analysis the tool 
for the task. He merely takes Dyer's approach one step further. "The basic 
technique is multiple regression analysis of the relationship between pupil 
performance and an array of pupil, teacher, and school characteristics- Howev€ 
the proposed methodolo,gy calls for two or three separate stages of analysis." 
(Barro, 1970:201) 

Summarizing Barro 's elaborate plan, we can say briefly, that, having 
obtained measures of pupil performance, the first stage of the analysis 
would entail separating the (Effects of schooling frcm those that can be 
attributed to pupil characteristics. Barro includes among the latter 
those variables Dyer terms "surrounding conditions." Given a measure 
of the extent to which schooling influenced output, the second stage 
would distinguish the contribution of: the individual from the effects 
of school and classroom variables beyond the control of the teacher- - 
including, in addition to those suggested by Dyer, characteristics of 

* c 

See above discussion of testing and Chapter V. 
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of the school's teaching staff and administrative and support personnel. 
A third stage might even distinghish among teacher characteristics which 
had a greater or lesser effect on pupil performance. Barro suggests that 
a similar approach could be used to determine the accountability of school 
administrators, although he notes that difficulties might arise where two 
or more persons shared responsibilities for rhe same task. This would 
apply as well to attempts to determine accountability of administrators 
of the district. Of course, if responsibilities could be distinguished 
or if they were merely jointly determined, the "output** being measured at 
this level is the quality of policy making and management. 

The data generated from the use of an accountability measure such 
as this could have valuable policy implications. Beyond the identification 
of relatively more or less effective schools and the indication of measure 
which might be taken to improve resource allocation — including evaluation 
of the relative effectiveness of educational technologies, data on the 
effects of distinct personnel characteristics could be used to refine 
personnel policies. Barro points out that such data could be used to 
guide personnel selection and assignment and might be applied to the 
development of a system of incentives — merit pay. 

Barro is aware of potential problems with this accountability 
measure: 1) omission of variables; 2) possible intercorrela^tion of 
variables, e.g. ^'better** teachers tend to end up teaching students of 
higher socio-economic status; 3) structural limitations of the model, 
e.g. relationships among pup.i i performances-variables not accounted for by 
considering output in each area of growth separately. And, of course, 



* 

"age, training, experience, ability and personality measures if available, 
ethnic and socio-economic background, etc.*' (Barro, 1970:202) 



Q He indicated that there are measures which might be taken to mitigate the 
FRIC effect of the last two problems. The first will, no doubt, always be a 
ma^ta possibility. 
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the model would need to be tested for validity. But he maintains that 
the benefits that might accrue from the use of such a system warrant 
the effort involved in its development. 

It is fair to say that the entire area of testing and measurement — 
determining what a student has learned and what factors accourit for that 
lea;.?iing~-is in somewhat of an embryonic stage, that the controversy 
generated by the accountability movement has stimulated interest in it 
and will likely contribute to its development. 
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